Titanic_Preprocess.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9da2cd5bf7d7126d3d3c2e4a0a94cd914e8b846e"
   },
   "source": [
    "# Titanic dataset preprocessing for classification\n",
    "\n",
    "In this tutorial we will learn a few ways of preprocessing a real world dataset such that it is suitable for feeding as an input to our favorite machine learning algorithms. \n",
    "\n",
    "Titanic dataset contains the information about the passangers onboard the famous Titanic ship. It also provides whether the passenger survived the incident, which occured on that fateful night, or not. Like any real world dataset all columns in the dataset are not numbers and there's a lot of missing information. Since, ML algorithms only crunch numbers, having text data makes it impossible to use it directly as an input to the machine learning algorithms which we have been studying. The objective of this tutorial is to introduce you to few methods which will help in converting various forms of data to numbers and use them as features to the ML algorithms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "# Most of the code in this notebook is borrowed from \n",
    "# (https://www.kaggle.com/sinakhorami/titanic/titanic-best-working-classifier)\n",
    "\n",
    "# The Pearson correlation map is taken from\n",
    "# https://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python\n",
    "\n",
    "\n",
    "# Refer https://www.datascience.com/blog/introduction-to-correlation-learn-data-science-tutorials\n",
    "# for a description of Pearson Correlation function\n",
    "\n",
    "\n",
    "# Import necessary packages/ modules\n",
    "\n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_uuid": "6382dfd6a3a34ecd1c9b4adce87ae550a1661751"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   PassengerId  Survived  Pclass  \\\n",
      "0            1         0       3   \n",
      "1            2         1       1   \n",
      "2            3         1       3   \n",
      "\n",
      "                                                Name     Sex   Age  SibSp  \\\n",
      "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
      "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
      "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
      "\n",
      "   Parch            Ticket     Fare Cabin Embarked  \n",
      "0      0         A/5 21171   7.2500   NaN        S  \n",
      "1      0          PC 17599  71.2833   C85        C  \n",
      "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 418 entries, 0 to 417\n",
      "Data columns (total 11 columns):\n",
      "PassengerId    418 non-null int64\n",
      "Pclass         418 non-null int64\n",
      "Name           418 non-null object\n",
      "Sex            418 non-null object\n",
      "Age            332 non-null float64\n",
      "SibSp          418 non-null int64\n",
      "Parch          418 non-null int64\n",
      "Ticket         418 non-null object\n",
      "Fare           417 non-null float64\n",
      "Cabin          91 non-null object\n",
      "Embarked       418 non-null object\n",
      "dtypes: float64(2), int64(4), object(5)\n",
      "memory usage: 36.0+ KB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# read the titanic dataset from the given csv file\n",
    "train = pd.read_csv('./input/train.csv')\n",
    "test = pd.read_csv('./input/test.csv')\n",
    "\n",
    "# print relevant information about the dataset\n",
    "print (train.head(3))\n",
    "print (train.info())\n",
    "print (test.info())\n",
    "\n",
    "# initialize random state to be used for splitting training and test set\n",
    "rand_state = 243"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "310ac4a9c1e44907210654fab57cafa2811e4246"
   },
   "source": [
    "# Influence of different passenger information on survival prediction\n",
    "\n",
    "The figure shows the correlation between different features in the dataset. Choose the best feature from this map that can help in predicting the survival rate of the passengers\n",
    "\n",
    "![](./PearsonCorrelation.png \"Pearson Correlation map\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Pclass  Survived\n",
      "0       1  0.629630\n",
      "1       2  0.472826\n",
      "2       3  0.242363\n"
     ]
    }
   ],
   "source": [
    "# group the survival rate by Pclass\n",
    "print (train[[\"Pclass\", \"Survived\"]].groupby(['Pclass'], as_index=False).mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ada496b70c7dfa341694e72a0c7d8d8d67ed416a"
   },
   "source": [
    "Anyone who has watched the Titanic movie will be well aware that, passengers travelling First class were allowed to get on the life-boats first. And the above statistics shows that more passengers travelling First class survived compared to others.\n",
    " \n",
    "## Now we will predict the survival rate based on Pclass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "c126f51ca6b03ef2c276ff1895b8a1b8deff894f"
   },
   "outputs": [],
   "source": [
    "# prepare the training dataset by mapping the columns to numerical values\n",
    "\n",
    "# make a copy of the training and the test set\n",
    "train_first_clf = train.copy()\n",
    "test_first_clf = test.copy()\n",
    "\n",
    "\n",
    "for dataset in [train_first_clf, test_first_clf]:\n",
    "    # add two columns FirstClass and SecondClass to the data and initialize with 0\n",
    "    dataset['FirstClass'] = 0.*dataset['Pclass']\n",
    "    dataset['SecondClass'] = 0.*dataset['Pclass']\n",
    "    \n",
    "    # fill the two newly added columns with appropriate data\n",
    "    dataset.loc[dataset['Pclass']==1, 'FirstClass'] = 1.\n",
    "    dataset.loc[dataset['Pclass']==2, 'SecondClass'] = 1.\n",
    "\n",
    "# select only the Pclass category as feature from the training data\n",
    "data_train = train_first_clf[['Survived','Pclass']]\n",
    "data_test = test_first_clf['Pclass']\n",
    "data_train = data_train.values\n",
    "data_test = data_test.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_uuid": "0d609b7c7a1954009a9fd1b17d5cb0efc10e1751"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             Classifier  Accuracy\n",
      "0    LogisticRegression  0.636771\n",
      "0            GaussianNB  0.636771\n",
      "0  KNeighborsClassifier  0.551570\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# import necessary libraries\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, log_loss\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# initialize the classifiers\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(3),\n",
    "    GaussianNB(),\n",
    "    LogisticRegression()]\n",
    "\n",
    "\n",
    "# initialize a table to store the classifier accuracies\n",
    "log_cols = [\"Classifier\", \"Accuracy\"]\n",
    "log \t = pd.DataFrame(columns=log_cols)\n",
    "\n",
    "\n",
    "# split the data into features and targets\n",
    "X = data_train[:, 1:]       # feature variables\n",
    "y = data_train[:, 0]        # targets\n",
    "\n",
    "acc_dict1 = {}\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=rand_state)\n",
    "for clf in classifiers:\n",
    "    name = clf.__class__.__name__\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    acc_dict1[name] = acc\n",
    "\n",
    "# visualize the test accuracy of different classifiers\n",
    "for clf in acc_dict1:\n",
    "    log_entry = pd.DataFrame([[clf, acc_dict1[clf]]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "\n",
    "print(log)\n",
    "plt.xlabel('Classifier')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.bar(np.arange(3), log['Accuracy'])\n",
    "plt.xticks(np.arange(3), acc_dict1.keys())\n",
    "plt.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What could be the next best feature to include to improve the accuracy of our classifier?\n",
    "\n",
    "Again, have a look at the Correlation map and decide on one more feature which could help in improving the accuracy of the classifier. Remember that in order to improve the accuracy we would need some extra information which is not conveyed by the already existing feature which is the Pclass\n",
    "\n",
    "![](./PearsonCorrelation.png \"Pearson Correlation map\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "83de2834c1b958b39331140d3d06e3ca34668bbd"
   },
   "source": [
    "Again, if we recall from the movie the passengers who had bought first class tickets were the first ones to be let onto the life boats. Hence, information about whether the passenger was male or female can help us in predicting the survival probability of the passenger along with the information whether the passenger was travelling first class, second class or third class.\n",
    "\n",
    "# Effect of Passenger class and Sex on survival rate "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_uuid": "a851df0efed0bfca99a4098485c45b75226dfd91"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      Sex  Survived\n",
      "0  female  0.742038\n",
      "1    male  0.188908\n"
     ]
    }
   ],
   "source": [
    "# group the data by sex of the passenger and check the survival rate\n",
    "print (train[['Sex', 'Survived']].groupby(['Sex'], as_index=False).mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "cbfed1536f779634c576afbc81cf039ad2659ffa"
   },
   "outputs": [],
   "source": [
    "## add Pclass to the features and classify\n",
    "train_second_clf = train.copy()\n",
    "test_second_clf = test.copy()\n",
    "for dataset in [train_second_clf, test_second_clf]:\n",
    "    # Mapping Sex to categorical values\n",
    "    dataset['Sex'] = dataset['Sex'].map( {'female':0., 'male': 1.} ).astype(float)\n",
    "    \n",
    "    # add two columns FirstClass and SecondClass to the data and initialize with 0\n",
    "    dataset['FirstClass'] = 0.*dataset['Pclass']\n",
    "    dataset['SecondClass'] = 0.*dataset['Pclass']\n",
    "    \n",
    "    # fill the two newly added columns with appropriate data\n",
    "    dataset.loc[dataset['Pclass']==1, 'FirstClass'] = 1.\n",
    "    dataset.loc[dataset['Pclass']==2, 'SecondClass'] = 1.\n",
    "\n",
    "# feature selection\n",
    "data_train = train_second_clf[['Survived','Sex', 'FirstClass', 'SecondClass']]\n",
    "data_test = test_second_clf[['Sex', 'FirstClass', 'SecondClass']]\n",
    "data_train = data_train.values\n",
    "data_test = data_test.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_uuid": "7a0ea2abc796c3dfd76788f2d19576c6b08336d9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             Classifier  Accuracy\n",
      "0    LogisticRegression  0.811659\n",
      "0            GaussianNB  0.771300\n",
      "0  KNeighborsClassifier  0.735426\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAG6pJREFUeJzt3XuYHVWZ7/Hvz0AIFwE5aXVOLiRgUCMCQhM5o+MwBjwBMUEFScYZiING0KDAgKJiDsY5HsUL5yARCYggCiEEBxttDchlUATscCfBQAxgkmE03A2KIfKeP9bqotjZ3Xt3p6t3Qv8+z9NPV61au+rde9eut2pV1SpFBGZmZgCvaHUAZma2+XBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlbYqtUB9NXIkSNj3LhxrQ7DzGyLcvvttz8WEW2N6m1xSWHcuHEsWbKk1WGYmW1RJD3STD03H5mZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrVJoUJE2RtFzSCkmn1Zk+VtINku6UdI+kQ6uMx8zMelfZHc2ShgHzgIOB1UCXpI6IWFaqdjqwMCLOlTQR6ATGVRXTuNN+UtWsh7yHv/zuVodgZgOgyiOFScCKiFgZEeuBBcC0mjoB7JiHdwL+s8J4zMysgSr7PhoFrCqNrwbeWlPnDOAaSScA2wMHVRiPmZk10OoTzTOAiyJiNHAocImkjWKSNEvSEklL1q5dO+hBmpkNFVUmhTXAmNL46FxWdiywECAibgFGACNrZxQR8yOiPSLa29oa9vxqZmb9VGVS6AImSBovaTgwHeioqfM7YDKApDeSkoIPBczMWqSypBARG4DZwGLgftJVRkslzZU0NVf7V+Ajku4GLgNmRkRUFZOZmfWu0ofsREQn6TLTctmc0vAy4G1VxmBmZs1r9YlmMzPbjDgpmJlZYYt7RrMNLb4LvTq+C93q8ZGCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4JvXjOzAeUbDqszGDcc+kjBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMysUGlSkDRF0nJJKySdVmf6WZLuyn8PSHqqynjMzKx3lV2SKmkYMA84GFgNdEnqyI/gBCAiTirVPwF4S1XxmJlZY1UeKUwCVkTEyohYDywApvVSfwZwWYXxmJlZA1UmhVHAqtL46ly2EUm7AuOB63uYPkvSEklL1q5dO+CBmplZsrmcaJ4OLIqIv9abGBHzI6I9Itrb2toGOTQzs6GjyqSwBhhTGh+dy+qZjpuOzMxarsqk0AVMkDRe0nDShr+jtpKkNwCvAm6pMBYzM2tCZUkhIjYAs4HFwP3AwohYKmmupKmlqtOBBRERVcViZmbNqbSX1IjoBDpryubUjJ9RZQxmZta8zeVEs5mZbQacFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs0KlSUHSFEnLJa2QdFoPdT4gaZmkpZIurTIeMzPrXWVPXpM0DJgHHAysBrokdUTEslKdCcBngLdFxJOSXl1VPGZm1liVRwqTgBURsTIi1gMLgGk1dT4CzIuIJwEi4g8VxmNmZg1UmRRGAatK46tzWdkewB6SbpZ0q6QpFcZjZmYNVNZ81IflTwAOBEYDN0l6c0Q8Va4kaRYwC2Ds2LGDHaOZ2ZBR5ZHCGmBMaXx0LitbDXRExPMR8RDwAClJvEREzI+I9ohob2trqyxgM7Ohrsqk0AVMkDRe0nBgOtBRU+cq0lECkkaSmpNWVhiTmZn1orKkEBEbgNnAYuB+YGFELJU0V9LUXG0x8LikZcANwKkR8XhVMZmZWe8qPacQEZ1AZ03ZnNJwACfnPzMzazHf0WxmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmhUqTgqQpkpZLWiHptDrTZ0paK+mu/PfhKuMxM7PeNUwKkk6Q9Kq+zljSMGAecAgwEZghaWKdqpdHxD7574K+LsfMzAZOM0cKrwG6JC3Me/5qct6TgBURsTIi1gMLgGn9DdTMzKrXMClExOnABOA7wEzgQUlfkrR7g5eOAlaVxlfnslrvl3SPpEWSxtSbkaRZkpZIWrJ27dpGIZuZWT81dU4hIgL4r/y3AXgVsEjSmZu4/KuBcRGxF3AtcHEPy58fEe0R0d7W1raJizQzs540c07hk5JuB84EbgbeHBHHA/sB7+/lpWuA8p7/6FxWiIjHI+IvefSCPE8zM2uRrZqoswvwvoh4pFwYES9IOqyX13UBEySNJyWD6cA/litI+puIeDSPTgXubzpyMzMbcM0khZ8CT3SPSNoReGNE3BYRPW7EI2KDpNnAYmAYcGFELJU0F1gSER3AJyRNJTVJPUE6Z2FmZi3STFI4F9i3NL6uTlldEdEJdNaUzSkNfwb4TFORmplZ5Zo50ax8ohlIzUY0l0zMzGwL00xSWCnpE5K2zn+fBFZWHZiZmQ2+ZpLCccDfkk4WrwbeCsyqMigzM2uNhs1AEfEH0pVDZmb2MtcwKUgaARwLvAkY0V0eEf9SYVxmZtYCzTQfXQK8FvifwH+QbkL7Y5VBmZlZazSTFF4XEZ8Hno2Ii4F3k84rmJnZy0wzSeH5/P8pSXsCOwGvri4kMzNrlWbuN5ifn6dwOtAB7AB8vtKozMysJXpNCpJeATwTEU8CNwG7DUpUZmbWEr02H+W7lz81SLGYmVmLNXNO4eeSTpE0RtIu3X+VR2ZmZoOumXMKR+X/Hy+VBW5KMjN72WnmjubxgxGImZm1XjN3NB9drzwivjfw4ZiZWSs103y0f2l4BDAZuANwUjAze5lppvnohPK4pJ2BBc3MXNIU4P+Rnrx2QUR8uYd67wcWAftHxJJm5m1mZgOvmauPaj0LNDzPIGkYMA84BJgIzJA0sU69VwKfBG7rRyxmZjaAmjmncDXpaiNISWQisLCJeU8CVkTEyjyfBcA0YFlNvS8CXwFObTJmMzOrSDPnFL5WGt4APBIRq5t43ShgVWm8+wE9BUn7AmMi4ieSnBTMzFqsmaTwO+DRiHgOQNK2ksZFxMObsuDchcY3gJlN1J1Fftrb2LFjN2WxZmbWi2bOKVwBvFAa/2sua2QNMKY0PjqXdXslsCdwo6SHgQOADknttTOKiPkR0R4R7W1tbU0s2szM+qOZpLBVRKzvHsnDw5t4XRcwQdJ4ScNJj/TsKM3n6YgYGRHjImIccCsw1VcfmZm1TjNJYa2kqd0jkqYBjzV6UURsAGYDi4H7gYURsVTS3PL8zMxs89HMOYXjgB9IOiePrwbq3uVcKyI6gc6asjk91D2wmXmamVl1mrl57bfAAZJ2yOPrKo/KzMxaomHzkaQvSdo5ItZFxDpJr5L0b4MRnJmZDa5mzikcEhFPdY/kp7AdWl1IZmbWKs0khWGStukekbQtsE0v9c3MbAvVzInmHwDXSfouINLNZhdXGZSZmbVGMyeavyLpbuAgUh9Ii4Fdqw7MzMwGX7O9pP6elBCOBN5Juu/AzMxeZno8UpC0BzAj/z0GXA4oIv5hkGIzM7NB1lvz0W+AXwCHRcQKAEknDUpUZmbWEr01H70PeBS4QdL5kiaTTjSbmdnLVI9JISKuiojpwBuAG4ATgVdLOlfSuwYrQDMzGzwNTzRHxLMRcWlEvIfU/fWdwKcrj8zMzAZdn57RHBFP5mcbTK4qIDMza50+JQUzM3t5c1IwM7OCk4KZmRWcFMzMrFBpUpA0RdJySSsknVZn+nGS7pV0l6RfSppYZTxmZta7ypKCpGHAPOAQYCIwo85G/9KIeHNE7AOcCXyjqnjMzKyxKo8UJgErImJlRKwHFgDTyhUi4pnS6PakTvfMzKxFmnmeQn+NAlaVxlcDb62tJOnjwMnAcFIPrBuRNAuYBTB27NgBD9TMzJKWn2iOiHkRsTvpLunTe6gzPyLaI6K9ra1tcAM0MxtCqkwKa4AxpfHRuawnC4DDK4zHzMwaqDIpdAETJI2XNByYDnSUK0iaUBp9N/BghfGYmVkDlZ1TiIgNkmaTHt85DLgwIpZKmgssiYgOYLakg4DngSeBY6qKx8zMGqvyRDMR0Ql01pTNKQ1/ssrlm5lZ37T8RLOZmW0+nBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7NCpUlB0hRJyyWtkHRaneknS1om6R5J10natcp4zMysd5UlBUnDgHnAIcBEYIakiTXV7gTaI2IvYBFwZlXxmJlZY1UeKUwCVkTEyohYDywAppUrRMQNEfGnPHorMLrCeMzMrIEqk8IoYFVpfHUu68mxwE8rjMfMzBrYqtUBAEj6J6Ad+Pseps8CZgGMHTt2ECMzMxtaqjxSWAOMKY2PzmUvIekg4HPA1Ij4S70ZRcT8iGiPiPa2trZKgjUzs2qTQhcwQdJ4ScOB6UBHuYKktwDnkRLCHyqMxczMmlBZUoiIDcBsYDFwP7AwIpZKmitpaq72VWAH4ApJd0nq6GF2ZmY2CCo9pxARnUBnTdmc0vBBVS7fzMz6xnc0m5lZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVmh0qQgaYqk5ZJWSDqtzvR3SLpD0gZJR1QZi5mZNVZZUpA0DJgHHAJMBGZImlhT7XfATODSquIwM7PmVfmM5knAiohYCSBpATANWNZdISIeztNeqDAOMzNrUpXNR6OAVaXx1bmszyTNkrRE0pK1a9cOSHBmZraxLeJEc0TMj4j2iGhva2trdThmZi9bVSaFNcCY0vjoXGZmZpupKpNCFzBB0nhJw4HpQEeFyzMzs01UWVKIiA3AbGAxcD+wMCKWSporaSqApP0lrQaOBM6TtLSqeMzMrLEqrz4iIjqBzpqyOaXhLlKzkpmZbQa2iBPNZmY2OJwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzQqVJQdIUScslrZB0Wp3p20i6PE+/TdK4KuMxM7PeVZYUJA0D5gGHABOBGZIm1lQ7FngyIl4HnAV8pap4zMyssSqPFCYBKyJiZUSsBxYA02rqTAMuzsOLgMmSVGFMZmbWiyqTwihgVWl8dS6rWyciNgBPA/+twpjMzKwXW7U6gGZImgXMyqPrJC1vZTyDaCTwWKuDaIbc8Adb0PcF/s6yofSd7dpMpSqTwhpgTGl8dC6rV2e1pK2AnYDHa2cUEfOB+RXFudmStCQi2lsdhzXH39eWx9/ZxqpsPuoCJkgaL2k4MB3oqKnTARyTh48Aro+IqDAmMzPrRWVHChGxQdJsYDEwDLgwIpZKmgssiYgO4DvAJZJWAE+QEoeZmbWIvGO++ZI0Kzed2RbA39eWx9/ZxpwUzMys4G4uzMysMCSSgqR1AzCP/y5pUS/Td5b0sWbr5zo35m5A7pbUJWmfTY1zIEmaK+mgVsfRX5JeI+lSSSsl3S7pFknvrXiZ7ZLO3oTXPyzpytL4EZIuysMzJa2VdJekpZIWSdpuAMLua4zrSsOHSnpA0q6SzpD0J0mvrle3l/l1Stq5QZ0bJW10lVD+TM7p63tohqRTJP0mf95dko7uLZZ+LqNYX3K3Pz/PyztK0gV1eoGo3JBICgMhIv4zIo7opcrOwMf6UL/bByNib+BbwFc3MUwA8uW9mywi5kTEzwdiXoMt3xl/FXBTROwWEfuRLmQYXeVyI2JJRHxiE2ezXy8bg8sjYp+IeBOwHjhqE5fVb5ImA2cDh0TEI7n4MeBf+zKfiDg0Ip4a6PgaUVJ3GyjpOOBgYFJE7ANMBga8t4Wa9eUtuWyfiLg8Ij4cEcuanVfuWmiTDdmkIGmcpOsl3SPpOkljc/nukm6VdK+kf+ve08n178vDb5L065zR75E0AfgysHsu+2pN/WGSvibpvlz/hDoh3ULpjm9J78p7tndIukLSDrn80Lz3cruksyX9OJefIekSSTeTrugaluPoysv8aK73N5JuynHeJ+nvct2L8vi9kk7KdS+SdEQenizpzjz9Qknb5PKHJX0hx3mvpDdU8HX1xzuB9RHx7e6CiHgkIr6Zv5tf5JjvkPS3AJIO7P488/g5kmbm4S9LWpY/y6/lsiPzZ3a3pJtq5yFpUv4O75T0K0mvz+UzJf1Q0s8kPSjpzJrYvw58rrc3lxP/9sCTm/Yx9Y+kdwDnA4dFxG9Lky4EjpK0S53X/FPpd3Ne90Ysr0Mj8/DnlY6efynpMkmnlGZxZH79A5L+rlQ+Rmnv/UFJ/6u0vJPz93OfpBNz2bg8/+8B9+XXbrTuA58Fjo+IZwAi4pmIuJgaks6VtETpyO0LpfI+rS9KR1ffB/bPn8/uKh2R9LI9eFjSVyTdARzZ8ItrRkS87P+AdXXKrgaOycP/AlyVh38MzMjDx3W/FhgH3JeHv0nawwcYDmxbnl6n/vGkvp22yuO75P83Au15+ETgS3l4JHATsH0e/zQwBxhB6hZkfC6/DPhxHj4DuB3YNo/PAk7Pw9sAS4DxpL24z+XyYcArgf2Aa0ux75z/X0S6f6R7uXvk8u8BJ+bhh4ET8vDHgAta/X3nWD4BnNXDtO2AEXl4AukSaYADuz/PPH4OMJPU9cpyXrwwo/vzuRcYVVNWzAPYsfSdHwRcmYdnAitJN2uOAB4BxpQ+z9cA9wOvy5//RaXXrQXuAn4P/AIY1oLP9nnSJeR71ZSfAZyS19UvlH97wBtJv7mt8/i3gKNL73kksH9+byPyevkgcErpt/L1PHwo8PPSZ/Jo/o62JW3o2/M6fS8pce4ALCXtiY8DXgAOyK/faN3P39uTvbz/G3nxd9v9Wx6Wy/fahPWlGC4vhx62B6XP7lMD+f0O2SMF4H8Al+bhS4C3l8qvyMOX1r4ouwX4rKRPA7tGxJ8bLOsg4LxI/TsREU+Upv1A0kOkPcN5uewAUs+yN0u6i3SD367AG4CVEfFQrndZzXI6SrG8Czg6v/420oo6gXRT4YcknQG8OSL+SNpA7Sbpm5KmAM/UzPf1wEMR8UAevxh4R2n6D/P/20k/us2OpHl5D60L2Bo4X9K9pO+6Ubvt08BzwHckvQ/4Uy6/GbhI0kdIG4VaOwFXKB0xngW8qTTtuoh4OiKeA5bx0i4I/kpqSvxMnXleHqk547WkjcypDWKvwvPAr0i9HNdzNnCMpFeWyiaTNsBdeZ2cDOxW87q3AT+KiOfyenl1zfSe1rNrI+LxvO7/kPRbfjvw7xHxbESsy+XdRxePRMStebjRut/IB/Je+p2k73ci/V9fetLT9qDb5X2MuVdDOSn0W0RcCkwF/gx0SnrnJszug6Qfx8WkIxBIbZfXRmpb3CciJkZETz/AsmdLwyLtwXfPY3xEXBMRN5E26GtIK+jREfEksDdpz+Q44II+voe/5P9/ZfPpT2spsG/3SER8nLQhagNOIu1p703aExueq23gpb+JEfm1G0i9/i4CDgN+lsuPA04nddVyu6Tazhy/CNwQEXsC7+meX/aX0nC9z+0S0vc0hjoi7SZezUuT82B5AfgAMEnSZ2snRjo/cCnw8VKxgItL6+PrI+KMPi63p/Ws9rr6RtfZF7+Teut+pCajdZJqk9ZLSBpPOjKaHBF7AT8hHYH2d33pcVH0vj14tqcX9sdQTgq/4sU7qD9IOhQHuBV4fx6ue4d1XllWRsTZwI9Ih4x/JB3y1nMt8NHcDkxte2v+gX8eOCC3yd8KvE3S63L97SXtQTok3U0vPoyot5OMi4HjJW2d57FHns+uwO8j4nzSxn/f3J77ioi4krTS7lszr+XAuO54gH8G/qOXZW8OrgdGSDq+VNZ9pc5OwKMR8QLpvXTvtT0CTFS6CmRnUhIht9/uFBGdpISydy7fPSJui4g5pGad2g34TrzY39fMvgQfEc+Tji5O6qXa24Hf9jK9MhHxJ+DdwAcl1dth+QbwUV7ceF8HHJHbzpG0S14Xy24G3iNpRP7MD2synIPz/LYFDs/z+QVwuKTtJG0PvJcXf+OFXtb9/wPMk7RjrreD8tVHJTuSNshPS3oN6dkxm7K+9KSn7UElNpe9uqptJ2l1afwbwAnAdyWdSvqCPpSnnQh8X9LnSBn+6Trz+wDwz5KeB/6LdC7gCUk356aCn/JiUxCkje8ewD35NeeT2qsLEfFnSV8HTo2IY5VOcF6mfEKXdH7gAaXLXn8m6VlSU1BPLiAdYt8hSfk9Hk5qtzw1x7EOOJp0gvu7evFKjJc0W0TEc5I+RGoK2Sov99tsxiIiJB0OnCXpU6T3/yypPfYO4Mr8I/9ZLiciVklaSGqXfojUJAAp2f9I0gjSXtvJufyrShcZiLTRuxv4+1IYZwIXSzqdtBfZV98hbajKjpL0dtIO3Wr6mGwGUl7npwA3SVpbM+0xSf9OTmoRsSx/Dtfk9ex50pHEI6XXdEnqAO4hHcndS/3fX61fA1eSriz7fkQsgXShRJ4G6QjgztIOVbee1v1zSeciuvJv5XnSBQDl93i3pDuB35DOud2cJ/V3fakrItbW2x4AD/T8qv7zHc01lK77/nPeqEwnnXSufThQy0jaISLW5Q39PODBiDir1XGZDYTS+r0d6eTqrIi4o9VxDSVD5UihL/YDzskb3adIVyZtTj4i6RhSO/idwHktjsdsIM1XukdjBOkchBPCIPORgpmZFYbyiWYzM6vhpGBmZgUnBTMzKzgp2JAk6bWSFkj6rVI/Up35Xo77BnAZRS+zSn1MLVXq12aUGvSga9YqPtFsQ06+suxXpKtbvp3L9ibdjHRuvgN5oJf5beCXEfH9frx2q+4uUsyq5iMFG4r+AXg+XtqD6t2kG5CAojfNej2p9rmXWUkfJt3w+EVJP9DGPejW6832wLz8DlLfSGaDwvcp2FC0J6lTtd78ATg43809gdT5YDvwj8DiiPjfSl0/bwfsQ+r9ck9ID1wqzygiLsh3If84IhbV3FV7LPB0ROyf71a9WdI1edq+wJ6lDhDNKuekYFbf1qSbGPchdcDW3ddMF3ChUp9SV0XEXZKKnjZJ3VlcU3eO9b0L2Ev5uRWk/pImkB6g82snBBtsbj6yoWgp6c713tTtSbWCXmbr9mabpw1o75dmzXBSsKHoemAbSbO6CyTtxUt7razbk6r618tsb+r2Ztv/t2a2adx8ZENO7uzwvcD/VXpQ0nOkJ1idWKr2Ler0pEo/epltoKfebM1awpekmplZwc1HZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzAr/H3Hv1y1A7lIfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# train and visualize the accuracy for different classifiers\n",
    "\n",
    "log_cols = [\"Classifier\", \"Accuracy\"]\n",
    "log \t = pd.DataFrame(columns=log_cols)\n",
    "\n",
    "X = data_train[:, 1:]\n",
    "y = data_train[:, 0]\n",
    "\n",
    "acc_dict2 = {}\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=rand_state)\n",
    "\n",
    "for clf in classifiers:\n",
    "    name = clf.__class__.__name__\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    acc_dict2[name] = acc\n",
    "\n",
    "for clf in acc_dict2:\n",
    "    log_entry = pd.DataFrame([[clf, acc_dict2[clf]]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "\n",
    "print(log)\n",
    "plt.xlabel('Classifier')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.bar(np.arange(3), log['Accuracy'])\n",
    "plt.xticks(np.arange(3), acc_dict2.keys())\n",
    "plt.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "755258c028dc644383f6ec4b382cd263f59b6bb2"
   },
   "source": [
    "# Exercise\n",
    "Use raw values of Pclass for classification i.e. use the PClass column from the data as it is and report the accuracy. Do not map the Pclass to binary values as done before. Also use the Sex of the passenger as the second feature for predicting the survival rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_uuid": "7af8a02e81f8194c2703609b5eee9df6c4ce6764"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             Classifier  Accuracy\n",
      "0    LogisticRegression  0.826816\n",
      "0            GaussianNB  0.826816\n",
      "0  KNeighborsClassifier  0.793296\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAG59JREFUeJzt3XuYHVWZ7/Hvz2AIFwWZtDonCSRg0IkICE3kjI7DGHACIkEFCaMDcdAICgoIioo5iOd4FC/Mg0QkIIIohJtig60RuYhGwG7uJBhoA5gwjjR3g2ISeeePtbpSbHbv3p3u6p3Qv8/z9NNVq1ZVvXvv2vVWrapaWxGBmZkZwEtaHYCZmW04nBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFTZpdQCDNX78+Jg8eXKrwzAz26jceuutj0ZE20D1NrqkMHnyZLq7u1sdhpnZRkXSQ83Uc/ORmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFTa6J5qHYvJJP251CC9aD37pHZUs159Zdar6zGzj5jMFMzMrOCmYmVmh0qQgaaakZZJ6JJ1UZ/q2kq6XdLukuyTtV2U8ZmbWWGVJQdIYYD6wLzANOFTStJpqJwOXRsQbgdnAN6uKx8zMBlblmcJ0oCcilkfEamAhMKumTgAvz8NbAf9VYTxmZjaAKpPCBGBFaXxlLis7BXi/pJVAJ3BMvQVJmiupW1J3b29vFbGamRmtv9B8KHB+REwE9gMulPSCmCJiQUS0R0R7W9uAPxxkZmbrqcrnFB4GJpXGJ+aysiOAmQARcZOkccB44JEK4zKzCvnZkuqMxLMlVZ4pdAFTJU2RNJZ0Ibmjps7vgRkAkv4BGAe4fcjMrEUqSwoRsRY4GlgE3Eu6y2iJpFMlHZCrfQL4kKQ7gYuBORERVcVkZmaNVdrNRUR0ki4gl8vmlYaXAm+uMgYzM2teqy80m5nZBsRJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzK1SaFCTNlLRMUo+kk+pMP13SHfnvPklPVhmPmZk1Vtkvr0kaA8wH9gFWAl2SOvKvrQEQEceV6h8DvLGqeMzMbGBVnilMB3oiYnlErAYWArMa1D+U9DvNZmbWIlUmhQnAitL4ylz2ApK2A6YA1/Uzfa6kbkndvb29wx6omZklG8qF5tnA5RHxt3oTI2JBRLRHRHtbW9sIh2ZmNnpUmRQeBiaVxifmsnpm46YjM7OWqzIpdAFTJU2RNJa04++orSTpdcArgJsqjMXMzJpQWVKIiLXA0cAi4F7g0ohYIulUSQeUqs4GFkZEVBWLmZk1p7JbUgEiohPorCmbVzN+SpUxmJlZ8zaUC81mZrYBcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKlSYFSTMlLZPUI+mkfuq8V9JSSUskXVRlPGZm1lhlv7wmaQwwH9gHWAl0SeqIiKWlOlOBTwNvjognJL2yqnjMzGxgVZ4pTAd6ImJ5RKwGFgKzaup8CJgfEU8ARMQjFcZjZmYDqDIpTABWlMZX5rKyHYEdJS2WdLOkmRXGY2ZmA6is+WgQ658K7AVMBG6U9IaIeLJcSdJcYC7AtttuO9IxmpmNGlWeKTwMTCqNT8xlZSuBjohYExEPAPeRksTzRMSCiGiPiPa2trbKAjYzG+2qTApdwFRJUySNBWYDHTV1riSdJSBpPKk5aXmFMZmZWQOVJYWIWAscDSwC7gUujYglkk6VdECutgh4TNJS4HrgxIh4rKqYzMyssUqvKUREJ9BZUzavNBzA8fnPzMxazE80m5lZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmhQGTgqRjJL1iJIIxM7PWauZM4VVAl6RLJc2UpGYXnusvk9Qj6aQ60+dI6pV0R/774GCCNzOz4TVgUoiIk4GpwLeBOcD9kr4oaYdG80kaA8wH9gWmAYdKmlan6iURsWv+O3ewL8DMzIZPU9cU8s9m/nf+Wwu8Arhc0mkNZpsO9ETE8ohYDSwEZg0xXjMzq1Az1xQ+LulW4DRgMfCGiDgK2B14T4NZJwArSuMrc1mt90i6S9LlkiY1H7qZmQ23Zs4UtgHeHRH/GhGXRcQagIh4Dth/iOu/CpgcETsD1wAX1Kskaa6kbkndvb29Q1ylmZn1p5mk8BPg8b4RSS+X9CaAiLi3wXwPA+Uj/4m5rBARj0XEX/PouaSzjxeIiAUR0R4R7W1tbU2EbGZm66OZpHAWsKo0viqXDaQLmCppiqSxwGygo1xB0t+XRg8AGiUZMzOr2CZN1FG+0AykZiNJA84XEWslHQ0sAsYA50XEEkmnAt0R0QF8TNIBpIvXj5PubjIzsxZpJiksl/Qx1p0dfARY3szCI6IT6Kwpm1ca/jTw6eZCNTOzqjXTfHQk8I+k6wErgTcBc6sMyszMWqOZZqBHSNcDzMzsRW7ApCBpHHAE8HpgXF95RPxHhXGZmVkLNNN8dCHwauBfgV+Qbi39U5VBmZlZazSTFF4TEZ8DnomIC4B3kK4rmJnZi0wzSWFN/v+kpJ2ArYBXVheSmZm1SjO3pC7Iv6dwMunhsy2Bz1UalZmZtUTDpCDpJcDTEfEEcCOw/YhEZWZmLdGw+Sh3evfJEYrFzMxarJlrCj+XdIKkSZK26furPDIzMxtxzVxTOCT//2ipLHBTkpnZi04zTzRPGYlAzMys9Zp5ovmweuUR8d3hD8fMzFqpmeajPUrD44AZwG2Ak4KZ2YtMM81Hx5THJW0NLKwsIjMza5lm7j6q9Qzg6wxmZi9CAyYFSVdJ6sh/VwPLgB82s3BJMyUtk9Qj6aQG9d4jKSS1Nx+6mZkNt2auKXy1NLwWeCgiVg40k6QxwHxgH9KP83RJ6oiIpTX1XgZ8HLil6ajNzKwSzTQf/R64JSJ+ERGLgcckTW5ivulAT0Qsj4jVpOsQs+rU+wLwZeDZ5kI2M7OqNJMULgOeK43/LZcNZAKwojS+MpcVJO0GTIqIHzexPDMzq1gzSWGTfKQPQB4eO9QV5872vg58oom6cyV1S+ru7e0d6qrNzKwfzSSFXkkH9I1ImgU82sR8DwOTSuMTc1mflwE7ATdIehDYE+iod7E5IhZERHtEtLe1tTWxajMzWx/NXGg+Evi+pDPz+Eqg7lPONbqAqZKmkJLBbODf+iZGxFPA+L5xSTcAJ0REd3Ohm5nZcGvm4bXfAXtK2jKPr2pmwRGxVtLRwCJgDHBeRCyRdCrQHREdQ4jbzMwq0EzfR18ETouIJ/P4K4BPRMTJA80bEZ1AZ03ZvH7q7tVMwGZmVp1mrins25cQAPKvsO1XXUhmZtYqzSSFMZI27RuRtBmwaYP6Zma2kWrmQvP3gWslfQcQMAe4oMqgzMysNZq50PxlSXcCe5N+cW0RsF3VgZmZ2chrtpfUP5ISwsHA24B7K4vIzMxapt8zBUk7Aofmv0eBSwBFxL+MUGxmZjbCGjUf/Rb4JbB/RPQASDpuRKIyM7OWaNR89G7gD8D1ks6RNIN0odnMzF6k+k0KEXFlRMwGXgdcDxwLvFLSWZLePlIBmpnZyBnwQnNEPBMRF0XEO0md2t0OfKryyMzMbMQN6jeaI+KJ3GPpjKoCMjOz1hlUUjAzsxc3JwUzMys4KZiZWcFJwczMCk4KZmZWqDQpSJopaZmkHkkn1Zl+pKS7Jd0h6VeSplUZj5mZNVZZUpA0BpgP7AtMAw6ts9O/KCLeEBG7AqcBX68qHjMzG1iVZwrTgZ6IWB4Rq4GFwKxyhYh4ujS6BaknVjMza5FmfmRnfU0AVpTGVwJvqq0k6aPA8cBYUrfcZmbWIi2/0BwR8yNiB1LXGSfXqyNprqRuSd29vb0jG6CZ2ShSZVJ4GJhUGp+Yy/qzEDiw3oTctUZ7RLS3tbUNY4hmZlZWZVLoAqZKmiJpLDAb6ChXkDS1NPoO4P4K4zEzswFUdk0hItZKOpr0m85jgPMiYomkU4HuiOgAjpa0N7AGeAI4vKp4zMxsYFVeaCYiOoHOmrJ5peGPV7l+MzMbnJZfaDYzsw2Hk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlaoNClImilpmaQeSSfVmX68pKWS7pJ0raTtqozHzMwaqywpSBoDzAf2BaYBh0qaVlPtdqA9InYGLgdOqyoeMzMbWJVnCtOBnohYHhGrgYXArHKFiLg+Iv6cR28GJlYYj5mZDaDKpDABWFEaX5nL+nME8JMK4zEzswFs0uoAACS9H2gH/rmf6XOBuQDbbrvtCEZmZja6VHmm8DAwqTQ+MZc9j6S9gc8CB0TEX+stKCIWRER7RLS3tbVVEqyZmVWbFLqAqZKmSBoLzAY6yhUkvRE4m5QQHqkwFjMza0JlSSEi1gJHA4uAe4FLI2KJpFMlHZCrfQXYErhM0h2SOvpZnJmZjYBKrylERCfQWVM2rzS8d5XrNzOzwfETzWZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCpUmBUkzJS2T1CPppDrT3yrpNklrJR1UZSxmZjawypKCpDHAfGBfYBpwqKRpNdV+D8wBLqoqDjMza16VP8c5HeiJiOUAkhYCs4ClfRUi4sE87bkK4zAzsyZV2Xw0AVhRGl+Zy8zMbAO1UVxoljRXUrek7t7e3laHY2b2olVlUngYmFQan5jLBi0iFkREe0S0t7W1DUtwZmb2QlUmhS5gqqQpksYCs4GOCtdnZmZDVFlSiIi1wNHAIuBe4NKIWCLpVEkHAEjaQ9JK4GDgbElLqorHzMwGVuXdR0REJ9BZUzavNNxFalYyM7MNwEZxodnMzEaGk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlaoNClImilpmaQeSSfVmb6ppEvy9FskTa4yHjMza6yypCBpDDAf2BeYBhwqaVpNtSOAJyLiNcDpwJerisfMzAZW5ZnCdKAnIpZHxGpgITCrps4s4II8fDkwQ5IqjMnMzBqoMilMAFaUxlfmsrp1ImIt8BTwdxXGZGZmDWzS6gCaIWkuMDePrpK0rJXxjKDxwKOtDqIZcsMfbESfF/gzy0bTZ7ZdM5WqTAoPA5NK4xNzWb06KyVtAmwFPFa7oIhYACyoKM4NlqTuiGhvdRzWHH9eGx9/Zi9UZfNRFzBV0hRJY4HZQEdNnQ7g8Dx8EHBdRESFMZmZWQOVnSlExFpJRwOLgDHAeRGxRNKpQHdEdADfBi6U1AM8TkocZmbWIvKB+YZL0tzcdGYbAX9eGx9/Zi/kpGBmZgV3c2FmZoVRkRQkrRqGZfwvSZc3mL61pI80Wz/XuSF3A3KnpC5Juw41zuEk6VRJe7c6jvUl6VWSLpK0XNKtkm6S9K6K19ku6YwhzP+gpCtK4wdJOj8Pz5HUK+kOSUskXS5p82EIe7AxrioN7yfpPknbSTpF0p8lvbJe3QbL65S09QB1bpD0gruE8nty5mBfQzMknSDpt/n97pJ0WKNY1nMdxfaSu/35eV7fIZLOrdMLROVGRVIYDhHxXxFxUIMqWwMfGUT9Pu+LiF2AbwJfGWKYAOTbe4csIuZFxM+HY1kjLT8ZfyVwY0RsHxG7k25kmFjleiOiOyI+NsTF7N5gZ3BJROwaEa8HVgOHDHFd603SDOAMYN+IeCgXPwp8YjDLiYj9IuLJ4Y5vIErq7gMlHQnsA0yPiF2BGcCw97ZQs728MZftGhGXRMQHI2Jps8vKXQsN2ahNCpImS7pO0l2SrpW0bS7fQdLNku6W9H/7jnRy/Xvy8Osl/SZn9LskTQW+BOyQy75SU3+MpK9KuifXP6ZOSDdReuJb0tvzke1tki6TtGUu3y8fvdwq6QxJV+fyUyRdKGkx6Y6uMTmOrrzOD+d6fy/pxhznPZL+Kdc9P4/fLem4XPd8SQfl4RmSbs/Tz5O0aS5/UNLnc5x3S3pdBR/X+ngbsDoivtVXEBEPRcQ38mfzyxzzbZL+EUDSXn3vZx4/U9KcPPwlSUvze/nVXHZwfs/ulHRj7TIkTc+f4e2Sfi3ptbl8jqQfSPqppPslnVYT+9eAzzZ6cTnxbwE8MbS3af1IeitwDrB/RPyuNOk84BBJ29SZ5/2l783ZfTuxvA2Nz8OfUzp7/pWkiyWdUFrEwXn++yT9U6l8ktLR+/2S/k9pfcfnz+ceScfmssl5+d8F7snzvmDbBz4DHBURTwNExNMRcQE1JJ0lqVvpzO3zpfJBbS9KZ1ffA/bI788OKp2RNNgfPCjpy5JuAw4e8INrRkS86P+AVXXKrgIOz8P/AVyZh68GDs3DR/bNC0wG7snD3yAd4QOMBTYrT69T/yhS306b5PFt8v8bgPY8fCzwxTw8HrgR2CKPfwqYB4wjdQsyJZdfDFydh08BbgU2y+NzgZPz8KZANzCFdBT32Vw+BngZsDtwTSn2rfP/80nPj/Std8dc/l3g2Dz8IHBMHv4IcG6rP+8cy8eA0/uZtjkwLg9PJd0iDbBX3/uZx88E5pC6XlnGuhsz+t6fu4EJNWXFMoCXlz7zvYEr8vAcYDnpYc1xwEPApNL7+SrgXuA1+f0/vzRfL3AH8Efgl8CYFry3a0i3kO9cU34KcELeVj9f/u4B/0D6zr00j38TOKz0mscDe+TXNi5vl/cDJ5S+K1/Lw/sBPy+9J3/In9FmpB19e96m7yYlzi2BJaQj8cnAc8Ceef4XbPv5c3uiweu/gXXf277v8phcvvMQtpdiuLwe+tkflN67Tw7n5ztqzxSA/w1clIcvBN5SKr8sD19UO1N2E/AZSZ8CtouIvwywrr2BsyP170REPF6a9n1JD5CODOfnsj1JPcsulnQH6QG/7YDXAcsj4oFc7+Ka9XSUYnk7cFie/xbShjqV9FDhBySdArwhIv5E2kFtL+kbkmYCT9cs97XAAxFxXx6/AHhrafoP8v9bSV+6DY6k+fkIrQt4KXCOpLtJn/VA7bZPAc8C35b0buDPuXwxcL6kD5F2CrW2Ai5TOmM8HXh9adq1EfFURDwLLOX5XRD8jdSU+Ok6y7wkUnPGq0k7mRMHiL0Ka4Bfk3o5rucM4HBJLyuVzSDtgLvyNjkD2L5mvjcDP4qIZ/N2eVXN9P62s2si4rG87f+A9F1+C/DDiHgmIlbl8r6zi4ci4uY8PNC2P5D35qP020mf7zTWf3vpT3/7gz6XDDLmhkZzUlhvEXERcADwF6BT0tuGsLj3kb4cF5DOQCC1XV4TqW1x14iYFhH9fQHLnikNi3QE37eMKRHxs4i4kbRDf5i0gR4WEU8Au5COTI4Ezh3ka/hr/v83Npz+tJYAu/WNRMRHSTuiNuA40pH2LqQjsbG52lqe/50Yl+ddS+r193Jgf+CnufxI4GRSVy23SqrtzPELwPURsRPwzr7lZX8tDdd73y4kfU6TqCPSYeJVPD85j5TngPcC0yV9pnZipOsDFwEfLRULuKC0Pb42Ik4Z5Hr7285q76sf6D774ntSb9uP1GS0SlJt0noeSVNIZ0YzImJn4MekM9D13V76XRWN9wfP9Dfj+hjNSeHXrHuC+n2kU3GAm4H35OG6T1jnjWV5RJwB/Ih0yvgn0ilvPdcAH87twNS2t+Yv+OeAPXOb/M3AmyW9JtffQtKOpFPS7bXux4gaXWRcBBwl6aV5GTvm5WwH/DEiziHt/HfL7bkviYgrSBvtbjXLWgZM7osH+HfgFw3WvSG4Dhgn6ahSWd+dOlsBf4iI50ivpe+o7SFgmtJdIFuTkgi5/XariOgkJZRdcvkOEXFLRMwjNevU7sC3Yl1/X3MGE3xErCGdXRzXoNpbgN81mF6ZiPgz8A7gfZLqHbB8Hfgw63be1wIH5bZzJG2Tt8WyxcA7JY3L7/n+TYazT17eZsCBeTm/BA6UtLmkLYB3se47Xmiw7f9/YL6kl+d6WyrffVTyctIO+SlJryL9dsxQtpf+9Lc/qMSGclRXtc0lrSyNfx04BviOpBNJH9AH8rRjge9J+iwpwz9VZ3nvBf5d0hrgv0nXAh6XtDg3FfyEdU1BkHa+OwJ35XnOIbVXFyLiL5K+BpwYEUcoXeC8WPmCLun6wH1Kt73+VNIzpKag/pxLOsW+TZLyazyQ1G55Yo5jFXAY6QL3d7TuToznNVtExLOSPkBqCtkkr/dbbMAiIiQdCJwu6ZOk1/8MqT32NuCK/CX/aS4nIlZIupTULv0AqUkAUrL/kaRxpKO243P5V5RuMhBpp3cn8M+lME4DLpB0MukocrC+TdpRlR0i6S2kA7qVDDLZDKe8zc8EbpTUWzPtUUk/JCe1iFia34ef5e1sDelM4qHSPF2SOoC7SGdyd1P/+1frN8AVpDvLvhcR3ZBulMjTIJ0B3F46oOrT37Z/FulaRFf+rqwh3QBQfo13Srod+C3pmtviPGl9t5e6IqK33v4AuK//udafn2iuoXTf91/yTmU26aJz7Y8DtYykLSNiVd7Rzwfuj4jTWx2X2XAobd+bky6uzo2I21od12gyWs4UBmN34My8032SdGfShuRDkg4ntYPfDpzd4njMhtMCpWc0xpGuQTghjDCfKZiZWWE0X2g2M7MaTgpmZlZwUjAzs4KTgo1Kkl4taaGk3yn1I9WZn+W4ZxjXUfQyq9TH1BKlfm0maIAedM1axReabdTJd5b9mnR3y7dy2S6kh5HOyk8gD/c6vwX8KiK+tx7zbtLXRYpZ1XymYKPRvwBr4vk9qN5JegAJKHrTrNeT6qB7mZX0QdIDj1+Q9H29sAfder3Z7pXX30HqG8lsRPg5BRuNdiJ1qtbII8A++WnuqaTOB9uBfwMWRcT/U+r6eXNgV1LvlztB+sGl8oIi4tz8FPLVEXF5zVO1RwBPRcQe+WnVxZJ+lqftBuxU6gDRrHJOCmb1vZT0EOOupA7Y+vqa6QLOU+pT6sqIuENS0dMmqTuLn9VdYn1vB3ZW/t0KUn9JU0k/oPMbJwQbaW4+stFoCenJ9Ubq9qRaQS+zdXuzzdOGtfdLs2Y4KdhodB2wqaS5fQWSdub5vVbW7UlV69fLbCN1e7Nd/5dmNjRuPrJRJ3d2+C7gP5V+KOlZ0i9YHVuq9k3q9KTKevQyO4D+erM1awnfkmpmZgU3H5mZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMr/A9QV9lTy0jErgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# make a copy of the original dataset\n",
    "# add code here\n",
    "train_third_clf = train.copy()\n",
    "test_third_clf = test.copy()\n",
    "\n",
    "for dataset in [train_third_clf, test_third_clf]:\n",
    "    # Map Sex (male , female) to categorical values\n",
    "    # add code here\n",
    "    dataset['Sex'] = dataset['Sex'].map( {'female':0., 'male': 1.} ).astype(float)\n",
    "    \n",
    "    \n",
    "# select the features Pclass, Sex and Survived from the training set \n",
    "# make sure that first column is Survived column\n",
    "# add code here\n",
    "\n",
    "data_train = train_third_clf[[\"Survived\",\"Pclass\",\"Sex\"]]\n",
    "data_test = test_third_clf[[\"Pclass\",\"Sex\"]]\n",
    "    \n",
    "data_train = data_train.values\n",
    "data_test = data_test.values\n",
    "\n",
    "\n",
    "log_cols = [\"Classifier\", \"Accuracy\"]\n",
    "log \t = pd.DataFrame(columns=log_cols)\n",
    "\n",
    "\n",
    "X = data_train[:, 1:]\n",
    "y = data_train[:, 0]\n",
    "\n",
    "acc_dict = {}\n",
    "\n",
    "# split the data into train and test\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=rand_state)\n",
    "\n",
    "# evaluate for different classifiers\n",
    "for clf in classifiers:\n",
    "    name = clf.__class__.__name__\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    acc_dict[name] = acc\n",
    "\n",
    "for clf in acc_dict:\n",
    "    log_entry = pd.DataFrame([[clf, acc_dict[clf]]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "\n",
    "print(log)\n",
    "plt.xlabel('Classifier')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.bar(np.arange(3), log['Accuracy'])\n",
    "plt.xticks(np.arange(3), acc_dict.keys())\n",
    "plt.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "606d8d7543a8b8efa149710a909460d0d323a3ef"
   },
   "source": [
    "# Assignment\n",
    "Convert the fare data into categorical data and use that information for classification\n",
    "For this check if all the rows in the data are non zero and/or non NULL. If there's any missing data fill it with some value (choose the values carefully, you can make use of the passenger class to decide on what values to fill)\n",
    "Once assured that there's no NULL data, convert the fare into three different categories. Here, you are binning continuous values into discrete categories. Note that although the values appear continuous a passenger travelling first class would have paid more than a passenger travelling thrid class. Bin the fares accordingly. Once the fares are binned use the column to classify and report the accuracy.\n",
    "\n",
    "### Functions that can be handy to handle the Fare data\n",
    "\".groupby\" as used previously for predicting the survival rate. Here you can use the function to calculate the median/ mean fare of the passengers and fill these values in places where data is missing\n",
    "\n",
    "\"pd.qcut\": This function bins the given column into \"N\" number of bins based on quantiles. If you have \"N=3\" then it bins the given column into top 33%, bottom 33% and the middle 33%. Refer to the documentation for more details.\n",
    "\n",
    "\".loc\": as used previously to select the required rows and columns and fill the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "## calculate the median fares for each of the passenger classes using .groupby\n",
    "# add code here\n",
    "median_fare = train['Fare'].groupby(train['Pclass']).median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "_uuid": "8e6363cf0e522e1d00ac1c90f6eaa951defb0600",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   CategoricalFare  Survived\n",
      "0  (-0.001, 8.662]  0.198052\n",
      "1    (8.662, 26.0]  0.402778\n",
      "2  (26.0, 512.329]  0.559322\n"
     ]
    }
   ],
   "source": [
    "## use pd.qcut to bin the fares into three categories and check the survival rate\n",
    "train['CategoricalFare'] = pd.qcut(train['Fare'], 3)\n",
    "print (train[['CategoricalFare', 'Survived']].groupby(['CategoricalFare'], as_index=False).mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "_uuid": "c668c966471a496b1b0013c523cfe5c341c96a58"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:16: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  app.launch_new_instance()\n",
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:17: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:18: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n"
     ]
    }
   ],
   "source": [
    "## make a copy of the original data\n",
    "train_fourth_clf = train.copy()\n",
    "test_fourth_clf = test.copy()\n",
    "\n",
    "## map the features into numerical values\n",
    "for dataset in [train_fourth_clf, test_fourth_clf]:\n",
    "    # Mapping Sex (male, female) to numerical values\n",
    "    # add code here\n",
    "    dataset['Sex'] = dataset['Sex'].map( {'female':0., 'male': 1.} ).astype(float)\n",
    "\n",
    "    # Fill zero/NULL values in fare according to the passenger class\n",
    "    # if the passenger is travelling first class \"and\" his fare information is missing then\n",
    "    # fill the fare information for that passenger with median fare for first class\n",
    "    # take care not to replace the existing non-zero/ non-NULL fare information for any of the passengers\n",
    "    # add code here\n",
    "    dataset['Fare'][(dataset['Fare']==0)  & (dataset['Pclass']==1)] = median_fare[1]\n",
    "    dataset['Fare'][(dataset['Fare']==0) & (dataset['Pclass']==2)] = median_fare[2]\n",
    "    dataset['Fare'][(dataset['Fare']==0) & (dataset['Pclass']==3)] = median_fare[3]\n",
    "    \n",
    "    # repeat the same for fares of passengers in second class and third class\n",
    "    # add code here\n",
    "    \n",
    "    # Grouping passengers into three categories based on fare\n",
    "    dataset.loc[ dataset['Fare'] <= 8.662, 'Fare'] \t\t\t\t\t\t        = 1\n",
    "    dataset.loc[(dataset['Fare'] > 8.662) & (dataset['Fare'] <= 26.0), 'Fare'] = 2\n",
    "    dataset.loc[ dataset['Fare'] > 26.0, 'Fare'] \t\t\t\t\t\t\t        = 3\n",
    "    dataset['Fare'] = dataset['Fare'].astype(float)\n",
    "    \n",
    "\n",
    "# Select the features \"Survived, Sex, Pclass, Fare\" from the given data\n",
    "# add code here\n",
    "data_train = train_third_clf[[\"Survived\",\"Sex\",\"Pclass\",\"Fare\"]]\n",
    "data_test = test_third_clf[[\"Sex\",\"Pclass\",\"Fare\"]]\n",
    "    \n",
    "data_train = data_train.values\n",
    "data_test = data_test.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "_uuid": "13d8cc8d36e6ec50eec373fa042b561c968ffcf2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             Classifier  Accuracy\n",
      "0    LogisticRegression  0.832402\n",
      "0            GaussianNB  0.821229\n",
      "0  KNeighborsClassifier  0.804469\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAG6ZJREFUeJzt3X2cHFWd7/HP10AIDwpyM+rePJCAQTciIAzRe3Vd1oA3ICaoIMm6C3HRCBoQEBQVczHu9So+cC8SkYAIohACuBg0GpCHRRF0wjMJBmIAk6yrAQMYFEnkt3+cM0Wl09PdM5maTjLf9+s1r6k6darq193V9es6VXVKEYGZmRnAS9odgJmZbTmcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVtmt3AL01fPjwGDNmTLvDMDPbqtx1111PRERHs3pbXVIYM2YMixcvbncYZmZbFUmPt1LPzUdmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZW2OruaN4cY878YbtD2GY99oV3tDsEM+sHPlIwM7PCoDpSsK2Pj+6q46M7q8dHCmZmVqg0KUiaJGmZpOWSzqwzfbSkWyTdI+l+SYdXGY+ZmTVWWVKQNASYAxwGjAemSRpfU+0sYH5EvAGYCny9qnjMzKy5Ko8UJgDLI2JFRDwPzAOm1NQJ4GV5eFfgPyqMx8zMmqjyRPMIYGVpfBXwxpo6ZwM3SDoJ2Bk4pMJ4zGwA+OKA6gzExQHtPtE8Dbg0IkYChwOXS9okJkkzJC2WtHjNmjUDHqSZ2WBRZVJYDYwqjY/MZWXHA/MBIuIOYBgwvHZBETE3IjojorOjo+kjRs3MrI+qTApdwDhJYyUNJZ1IXlBT5zfARABJf0tKCj4UMDNrk8qSQkRsAGYCi4CHSFcZLZE0W9LkXO1jwAcl3QdcCUyPiKgqJjMza6zSO5ojYiGwsKZsVml4KfDmKmMwM7PWtftEs5mZbUGcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs0KlSUHSJEnLJC2XdGad6edKujf/PSzpqSrjMTOzxip78pqkIcAc4FBgFdAlaUF+2hoAEXFqqf5JwBuqisfMzJqr8khhArA8IlZExPPAPGBKg/rTSM9pNjOzNqkyKYwAVpbGV+WyTUjaAxgL3FxhPGZm1sSWcqJ5KnBNRPy13kRJMyQtlrR4zZo1AxyamdngUWVSWA2MKo2PzGX1TKVB01FEzI2Izojo7Ojo6McQzcysrMqk0AWMkzRW0lDSjn9BbSVJrwVeDtxRYSxmZtaCypJCRGwAZgKLgIeA+RGxRNJsSZNLVacC8yIiqorFzMxaU9klqQARsRBYWFM2q2b87CpjMDOz1m0pJ5rNzGwL4KRgZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVKk0KkiZJWiZpuaQze6jzXklLJS2RdEWV8ZiZWWOVPXlN0hBgDnAosArokrQgIpaW6owDPgm8OSLWSnpFVfGYmVlzVR4pTACWR8SKiHgemAdMqanzQWBORKwFiIjfVxiPmZk1UWVSGAGsLI2vymVlewN7S7pd0p2SJlUYj5mZNVFZ81Ev1j8OOBgYCdwm6fUR8VS5kqQZwAyA0aNHD3SMZmaDRpVHCquBUaXxkbmsbBWwICLWR8SjwMOkJLGRiJgbEZ0R0dnR0VFZwGZmg12VSaELGCdprKShwFRgQU2d60hHCUgaTmpOWlFhTGZm1kBlSSEiNgAzgUXAQ8D8iFgiabakybnaIuBJSUuBW4AzIuLJqmIyM7PGKj2nEBELgYU1ZbNKwwGclv/MzKzNfEezmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMys4KZiZWcFJwczMCk2TgqSTJL18IIIxM7P2auVI4ZVAl6T5kiZJUtVBmZlZezRNChFxFum5yd8EpgOPSPq8pL2azZuTyDJJyyWdWWf6dElrJN2b/z7Qh9dgZmb9pKVzCvkJaf+Z/zYALweukXROT/NIGgLMAQ4DxgPTJI2vU/WqiNg//13c2xdgZmb9p5VzCh+VdBdwDnA78PqIOBE4EHhPg1knAMsjYkVEPA/MA6b0Q8xmZlaRVp7RvDvw7oh4vFwYES9IOqLBfCOAlaXxVcAb69R7j6S3Ag8Dp0bEytoKkmYAMwBGjx7dQshmZtYXrTQf/Qj4Q/eIpJdJeiNARDy0meu/HhgTEfsCNwKX1asUEXMjojMiOjs6OjZzlWZm1pNWksIFwLrS+Lpc1sxqYFRpfGQuK0TEkxHxlzx6MalJyszM2qSVpKB8ohlIzUa01uzUBYyTNFbSUGAqsGCjBUt/UxqdDGzukYeZmW2GVpLCCkknS9o+/30UWNFspojYAMwEFpF29vMjYomk2ZIm52onS1oi6T7gZNIlr2Zm1iat/OI/ATgPOAsI4CbySd9mImIhsLCmbFZp+JPAJ1sN1szMqtU0KUTE70lNP2Zmto1rmhQkDQOOB14HDOsuj4h/qTAuMzNrg1bOKVwOvAr4X8C/k64i+mOVQZmZWXu0khReHRGfAZ6NiMuAd1D/JjQzM9vKtZIU1uf/T0naB9gVeEV1IZmZWbu0cvXR3Pw8hbNI9xnsAnym0qjMzKwtGiYFSS8BnomItcBtwJ4DEpWZmbVFw+ajfPfyxwcoFjMza7NWzin8RNLpkkZJ2r37r/LIzMxswLVyTuGY/P8jpbLATUlmZtucVu5oHjsQgZiZWfu1ckfzsfXKI+Lb/R+OmZm1UyvNRweVhocBE4G7AScFM7NtTCvNRyeVxyXtRnrespmZbWNaufqo1rOAzzOYmW2DWjmncD3paiNISWQ8ML/KoMzMrD1aOafw5dLwBuDxiFjVysIlTQL+PzAEuDgivtBDvfcA1wAHRcTiVpZtZmb9r5Wk8BvgtxHxHICkHSWNiYjHGs0kaQgwBzgUWAV0SVoQEUtr6r0U+Cjwiz7Eb2Zm/aiVcwpXAy+Uxv+ay5qZACyPiBUR8Tzp5PSUOvU+B3wReK6FZZqZWYVaSQrb5Z06AHl4aAvzjQBWlsZX5bKCpAOAURHxw0YLkjRD0mJJi9esWdPCqs3MrC9aSQprJE3uHpE0BXhic1ece2D9KvCxZnUjYm5EdEZEZ0dHx+au2szMetDKOYUTgO9KOj+PrwLq3uVcYzUwqjQ+Mpd1eymwD3CrJEiP/FwgabJPNpuZtUcrN6/9GniTpF3y+LoWl90FjJM0lpQMpgL/WFru08Dw7nFJtwKnOyGYmbVP0+YjSZ+XtFtErIuIdZJeLulfm80XERuAmcAi4CFgfkQskTS73BxlZmZbjlaajw6LiE91j0TEWkmHkx7P2VBELAQW1pTN6qHuwS3EYmZmFWrlRPMQSTt0j0jaEdihQX0zM9tKtXKk8F3gJknfAgRMBy6rMigzM2uPVk40f1HSfcAhpD6QFgF7VB2YmZkNvFZ7Sf0dKSEcDbyNdOLYzMy2MT0eKUjaG5iW/54ArgIUEf8wQLGZmdkAa9R89Cvgp8AREbEcQNKpAxKVmZm1RaPmo3cDvwVukXSRpImkE81mZraN6jEpRMR1ETEVeC1wC3AK8ApJF0h6+0AFaGZmA6fpieaIeDYiroiId5L6L7oH+ETlkZmZ2YDr1TOaI2Jt7rF0YlUBmZlZ+/QqKZiZ2bbNScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKxQaVKQNEnSMknLJZ1ZZ/oJkh6QdK+kn0kaX2U8ZmbWWGVJQdIQYA5wGDAemFZnp39FRLw+IvYHzgG+WlU8ZmbWXJVHChOA5RGxIiKeB+YBU8oVIuKZ0ujOpO65zcysTVp58lpfjQBWlsZXAW+srSTpI8BpwFDSsxo2IWkGMANg9OjR/R6omZklbT/RHBFzImIvUn9KZ/VQZ25EdEZEZ0dHx8AGaGY2iFSZFFYDo0rjI3NZT+YBR1YYj5mZNVFlUugCxkkaK2koMBVYUK4gaVxp9B3AIxXGY2ZmTVR2TiEiNkiaCSwChgCXRMQSSbOBxRGxAJgp6RBgPbAWOK6qeMzMrLkqTzQTEQuBhTVls0rDH61y/WZm1jttP9FsZmZbDicFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFSpNCpImSVomabmkM+tMP03SUkn3S7pJ0h5VxmNmZo1VlhQkDQHmAIcB44FpksbXVLsH6IyIfYFrgHOqisfMzJqr8khhArA8IlZExPPAPGBKuUJE3BIRf8qjdwIjK4zHzMyaqDIpjABWlsZX5bKeHA/8qMJ4zMysiUqf0dwqSf8EdAJ/38P0GcAMgNGjRw9gZGZmg0uVRwqrgVGl8ZG5bCOSDgE+DUyOiL/UW1BEzI2Izojo7OjoqCRYMzOrNil0AeMkjZU0FJgKLChXkPQG4EJSQvh9hbGYmVkLKksKEbEBmAksAh4C5kfEEkmzJU3O1b4E7AJcLeleSQt6WJyZmQ2ASs8pRMRCYGFN2azS8CFVrt/MzHrHdzSbmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZwUnBzMwKTgpmZlZwUjAzs4KTgpmZFZwUzMys4KRgZmYFJwUzMytUmhQkTZK0TNJySWfWmf5WSXdL2iDpqCpjMTOz5ipLCpKGAHOAw4DxwDRJ42uq/QaYDlxRVRxmZta6Kh/HOQFYHhErACTNA6YAS7srRMRjedoLFcZhZmYtqrL5aASwsjS+KpeZmdkWaqs40SxphqTFkhavWbOm3eGYmW2zqkwKq4FRpfGRuazXImJuRHRGRGdHR0e/BGdmZpuqMil0AeMkjZU0FJgKLKhwfWZmtpkqSwoRsQGYCSwCHgLmR8QSSbMlTQaQdJCkVcDRwIWSllQVj5mZNVfl1UdExEJgYU3ZrNJwF6lZyczMtgBbxYlmMzMbGE4KZmZWcFIwM7OCk4KZmRWcFMzMrOCkYGZmBScFMzMrOCmYmVnBScHMzApOCmZmVnBSMDOzgpOCmZkVnBTMzKzgpGBmZgUnBTMzKzgpmJlZodKkIGmSpGWSlks6s870HSRdlaf/QtKYKuMxM7PGKksKkoYAc4DDgPHANEnja6odD6yNiFcD5wJfrCoeMzNrrsojhQnA8ohYERHPA/OAKTV1pgCX5eFrgImSVGFMZmbWQJVJYQSwsjS+KpfVrRMRG4Cngf9WYUxmZtbAdu0OoBWSZgAz8ug6ScvaGc8AGg480e4gWiE3/MFW9HmBP7NsMH1me7RSqcqksBoYVRofmcvq1VklaTtgV+DJ2gVFxFxgbkVxbrEkLY6IznbHYa3x57X18We2qSqbj7qAcZLGShoKTAUW1NRZAByXh48Cbo6IqDAmMzNroLIjhYjYIGkmsAgYAlwSEUskzQYWR8QC4JvA5ZKWA38gJQ4zM2sT+Yf5lkvSjNx0ZlsBf15bH39mm3JSMDOzgru5MDOzwqBICpLW9cMy/rukaxpM303Sh1utn+vcmrsBuU9Sl6T9NzfO/iRptqRD2h1HX0l6paQrJK2QdJekOyS9q+J1dko6bzPmf0zStaXxoyRdmoenS1oj6V5JSyRdI2mnfgi7tzGuKw0fLulhSXtIOlvSnyS9ol7dBstbKGm3JnVulbTJVUL5PTm/t6+hFZJOl/Sr/H53STq2USx9XEexveRuf36S13eMpIvr9AJRuUGRFPpDRPxHRBzVoMpuwId7Ub/b+yJiP+DrwJc2M0wA8uW9my0iZkXET/pjWQMt3xl/HXBbROwZEQeSLmQYWeV6I2JxRJy8mYs5sMHO4KqI2D8iXgc8DxyzmevqM0kTgfOAwyLi8Vz8BPCx3iwnIg6PiKf6O75mlNTdB0o6ATgUmBAR+wMTgX7vbaFme3lDLts/Iq6KiA9ExNJWl5W7FtpsgzYpSBoj6WZJ90u6SdLoXL6XpDslPSDpX7t/6eT6D+bh10n6Zc7o90saB3wB2CuXfamm/hBJX5b0YK5/Up2Q7qB0x7ekt+dftndLulrSLrn88Pzr5S5J50n6QS4/W9Llkm4nXdE1JMfRldf5oVzvbyTdluN8UNLf5bqX5vEHJJ2a614q6ag8PFHSPXn6JZJ2yOWPSfpsjvMBSa+t4OPqi7cBz0fEN7oLIuLxiPha/mx+mmO+W9L/BJB0cPf7mcfPlzQ9D39B0tL8Xn45lx2d37P7JN1WuwxJE/JneI+kn0t6TS6fLul7kn4s6RFJ59TE/hXg041eXE78OwNrN+9t6htJbwUuAo6IiF+XJl0CHCNp9zrz/FPpe3Nh904sb0PD8/BnlI6efybpSkmnlxZxdJ7/YUl/VyofpfTr/RFJ/7u0vtPy5/OgpFNy2Zi8/G8DD+Z5N9n2gU8BJ0bEMwAR8UxEXEYNSRdIWqx05PbZUnmvthelo6vvAAfl92cvlY5IGuwPHpP0RUl3A0c3/eBaERHb/B+wrk7Z9cBxefhfgOvy8A+AaXn4hO55gTHAg3n4a6Rf+ABDgR3L0+vUP5HUt9N2eXz3/P9WoDMPnwJ8Pg8PB24Dds7jnwBmAcNI3YKMzeVXAj/Iw2cDdwE75vEZwFl5eAdgMTCW9Cvu07l8CPBS4EDgxlLsu+X/l5LuH+le7965/NvAKXn4MeCkPPxh4OJ2f945lpOBc3uYthMwLA+PI10iDXBw9/uZx88HppO6XlnGixdmdL8/DwAjasqKZQAvK33mhwDX5uHpwArSzZrDgMeBUaX385XAQ8Cr8/t/aWm+NcC9wO+AnwJD2vDeriddQr5vTfnZwOl5W/1s+bsH/C3pO7d9Hv86cGzpNQ8HDsqvbVjeLh8BTi99V76Shw8HflJ6T36bP6MdSTv6zrxNP0BKnLsAS0i/xMcALwBvyvNvsu3nz21tg9d/Ky9+b7u/y0Ny+b6bsb0Uw+X10MP+oPTefbw/P99Be6QA/A/gijx8OfCWUvnVefiK2pmyO4BPSfoEsEdE/LnJug4BLozUvxMR8YfStO9KepT0y3BOLnsTqWfZ2yXdS7rBbw/gtcCKiHg017uyZj0LSrG8HTg2z/8L0oY6jnRT4fslnQ28PiL+SNpB7Snpa5ImAc/ULPc1wKMR8XAevwx4a2n69/L/u0hfui2OpDn5F1oXsD1wkaQHSJ91s3bbp4HngG9Kejfwp1x+O3CppA+Sdgq1dgWuVjpiPBd4XWnaTRHxdEQ8Byxl4y4I/kpqSvxknWVeFak541WkncwZTWKvwnrg56Rejus5DzhO0ktLZRNJO+CuvE1OBPasme/NwPcj4rm8XV5fM72n7ezGiHgyb/vfI32X3wL8W0Q8GxHrcnn30cXjEXFnHm627Tfz3vwr/R7S5zuevm8vPelpf9Dtql7G3NBgTgp9FhFXAJOBPwMLJb1tMxb3PtKX4zLSEQiktssbI7Ut7h8R4yOipy9g2bOlYZF+wXcvY2xE3BARt5F26KtJG+ixEbEW2I/0y+QE4OJevoa/5P9/ZcvpT2sJcED3SER8hLQj6gBOJf3S3o/0S2xorraBjb8Tw/K8G0i9/l4DHAH8OJefAJxF6qrlLkm1nTl+DrglIvYB3tm9vOwvpeF679vlpM9pFHVE+pl4PRsn54HyAvBeYIKkT9VOjHR+4ArgI6ViAZeVtsfXRMTZvVxvT9tZ7XX1za6zL74n9bb9SE1G6yTVJq2NSBpLOjKaGBH7Aj8kHYH2dXvpcVU03h8829OMfTGYk8LPefEO6veRDsUB7gTek4fr3mGdN5YVEXEe8H3SIeMfSYe89dwIfCi3A1Pb3pq/4J8B3pTb5O8E3izp1bn+zpL2Jh2S7qkXH0bU6CTjIuBESdvnZeydl7MH8LuIuIi08z8gt+e+JCKuJW20B9Qsaxkwpjse4J+Bf2+w7i3BzcAwSSeWyrqv1NkV+G1EvEB6Ld2/2h4HxitdBbIbKYmQ2293jYiFpISyXy7fKyJ+ERGzSM06tTvwXXmxv6/pvQk+ItaTji5ObVDtLcCvG0yvTET8CXgH8D5J9X6wfBX4EC/uvG8Cjspt50jaPW+LZbcD75Q0LL/nR7QYzqF5eTsCR+bl/BQ4UtJOknYG3sWL3/FCg23//wJzJL0s19tF+eqjkpeRdshPS3ol6dkxm7O99KSn/UEltpRfdVXbSdKq0vhXgZOAb0k6g/QBvT9POwX4jqRPkzL803WW917gnyWtB/6TdC7gD5Juz00FP+LFpiBIO9+9gfvzPBeR2qsLEfFnSV8BzoiI45VOcF6pfEKXdH7gYaXLXn8s6VlSU1BPLiYdYt8tSfk1Hklqtzwjx7EOOJZ0gvtbevFKjI2aLSLiOUnvJzWFbJfX+w22YBERko4EzpX0cdLrf5bUHns3cG3+kv84lxMRKyXNJ7VLP0pqEoCU7L8vaRjpV9tpufxLShcZiLTTuw/4+1IY5wCXSTqL9Cuyt75J2lGVHSPpLaQfdKvoZbLpT3mbnwTcJmlNzbQnJP0bOalFxNL8PtyQt7P1pCOJx0vzdElaANxPOpJ7gPrfv1q/BK4lXVn2nYhYDOlCiTwN0hHAPaUfVN162vYvIJ2L6MrflfWkCwDKr/E+SfcAvyKdc7s9T+rr9lJXRKyptz8AHu55rr7zHc01lK77/nPeqUwlnXSufThQ20jaJSLW5R39HOCRiDi33XGZ9YfS9r0T6eTqjIi4u91xDSaD5UihNw4Ezs873adIVyZtST4o6ThSO/g9wIVtjsesP81VukdjGOkchBPCAPORgpmZFQbziWYzM6vhpGBmZgUnBTMzKzgp2KAk6VWS5kn6tVI/UgvzvRwP9uM6il5mlfqYWqLUr80INelB16xdfKLZBp18ZdnPSVe3fCOX7Ue6GemCfAdyf6/zG8DPIuI7fZh3u+4uUsyq5iMFG4z+AVgfG/egeh/pBiSg6E2zXk+qve5lVtIHSDc8fk7Sd7VpD7r1erM9OK9/AalvJLMB4fsUbDDah9SpWiO/Bw7Nd3OPI3U+2An8I7AoIv6PUtfPOwH7k3q/3AfSA5fKC4qIi/NdyD+IiGtq7qo9Hng6Ig7Kd6veLumGPO0AYJ9SB4hmlXNSMKtve9JNjPuTOmDr7mumC7hEqU+p6yLiXklFT5uk7ixuqLvE+t4O7Kv83ApSf0njSA/Q+aUTgg00Nx/ZYLSEdOd6I3V7Uq2gl9m6vdnmaf3a+6VZK5wUbDC6GdhB0ozuAkn7snGvlXV7UlXfepltpG5vtn1/aWabx81HNujkzg7fBfw/pQclPUd6gtUppWpfp05PqvShl9kmeurN1qwtfEmqmZkV3HxkZmYFJwUzMys4KZiZWcFJwczMCk4KZmZWcFIwM7OCk4KZmRWcFMzMrPBfdjrNowU283EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "log_cols = [\"Classifier\", \"Accuracy\"]\n",
    "log \t = pd.DataFrame(columns=log_cols)\n",
    "\n",
    "X = data_train[:, 1:]\n",
    "y = data_train[:, 0]\n",
    "\n",
    "acc_dict = {}\n",
    "\n",
    "# split the data into train and test\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=rand_state)\n",
    "\n",
    "# evaluate for different classifiers\n",
    "for clf in classifiers:\n",
    "    name = clf.__class__.__name__\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    acc_dict[name] = acc\n",
    "\n",
    "for clf in acc_dict:\n",
    "    log_entry = pd.DataFrame([[clf, acc_dict[clf]]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "\n",
    "print(log)\n",
    "plt.xlabel('Classifier')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.bar(np.arange(3), log['Accuracy'])\n",
    "plt.xticks(np.arange(3), acc_dict.keys())\n",
    "plt.plot()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Is Fare a good feature to include in the classification?\n",
    "\n",
    "Ticket fare paid by a passenger seems to be a good feature for classification if considered independently. However, when we already have the information about the passenger class in which the passenger was travelling this feature becomes somewhat redundant. This is because, a passenger travelling first class would have certainly paid more compared to another passenger travelling third class. Even from the Pearson Correlation map we can see that the feature \"Pclass\" is highly correlated with the feature \"Fare\"\n",
    "\n",
    "![](./PearsonCorrelation.png \"Pearson Correlation map\")\n",
    "\n",
    "When we have a lot of features and very less data this is another way of reducing the dimensionality without losing information. If we remove the columns which are highly correlated then, we reduce the dimensionality without sacrificing much on the information"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}