FakeAdClassification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression Project \n",
    "\n",
    "In this project we will be working with a fake advertising data set, indicating whether or not a particular internet user clicked on an Advertisement. We will try to create a model that will predict whether or not they will click on an ad based off the features of that user.\n",
    "\n",
    "This data set contains the following features:\n",
    "\n",
    "* 'Daily Time Spent on Site': consumer time on site in minutes\n",
    "* 'Age': cutomer age in years\n",
    "* 'Area Income': Avg. Income of geographical area of consumer\n",
    "* 'Daily Internet Usage': Avg. minutes a day consumer is on the internet\n",
    "* 'Ad Topic Line': Headline of the advertisement\n",
    "* 'City': City of consumer\n",
    "* 'Male': Whether or not consumer was male\n",
    "* 'Country': Country of consumer\n",
    "* 'Timestamp': Time at which consumer clicked on Ad or closed window\n",
    "* 'Clicked on Ad': 0 or 1 indicated clicking on Ad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the Data\n",
    "**Read in the advertising.csv file and set it to a data frame called ad_data.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "ad_data = pd.read_csv(\"advertising.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Check the head of ad_data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Daily Time Spent on Site</th>\n",
       "      <th>Age</th>\n",
       "      <th>Area Income</th>\n",
       "      <th>Daily Internet Usage</th>\n",
       "      <th>Ad Topic Line</th>\n",
       "      <th>City</th>\n",
       "      <th>Male</th>\n",
       "      <th>Country</th>\n",
       "      <th>Timestamp</th>\n",
       "      <th>Clicked on Ad</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>68.95</td>\n",
       "      <td>35</td>\n",
       "      <td>61833.90</td>\n",
       "      <td>256.09</td>\n",
       "      <td>Cloned 5thgeneration orchestration</td>\n",
       "      <td>Wrightburgh</td>\n",
       "      <td>0</td>\n",
       "      <td>Tunisia</td>\n",
       "      <td>2016-03-27 00:53:11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>80.23</td>\n",
       "      <td>31</td>\n",
       "      <td>68441.85</td>\n",
       "      <td>193.77</td>\n",
       "      <td>Monitored national standardization</td>\n",
       "      <td>West Jodi</td>\n",
       "      <td>1</td>\n",
       "      <td>Nauru</td>\n",
       "      <td>2016-04-04 01:39:02</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>69.47</td>\n",
       "      <td>26</td>\n",
       "      <td>59785.94</td>\n",
       "      <td>236.50</td>\n",
       "      <td>Organic bottom-line service-desk</td>\n",
       "      <td>Davidton</td>\n",
       "      <td>0</td>\n",
       "      <td>San Marino</td>\n",
       "      <td>2016-03-13 20:35:42</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>74.15</td>\n",
       "      <td>29</td>\n",
       "      <td>54806.18</td>\n",
       "      <td>245.89</td>\n",
       "      <td>Triple-buffered reciprocal time-frame</td>\n",
       "      <td>West Terrifurt</td>\n",
       "      <td>1</td>\n",
       "      <td>Italy</td>\n",
       "      <td>2016-01-10 02:31:19</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>68.37</td>\n",
       "      <td>35</td>\n",
       "      <td>73889.99</td>\n",
       "      <td>225.58</td>\n",
       "      <td>Robust logistical utilization</td>\n",
       "      <td>South Manuel</td>\n",
       "      <td>0</td>\n",
       "      <td>Iceland</td>\n",
       "      <td>2016-06-03 03:36:18</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Daily Time Spent on Site  Age  Area Income  Daily Internet Usage  \\\n",
       "0                     68.95   35     61833.90                256.09   \n",
       "1                     80.23   31     68441.85                193.77   \n",
       "2                     69.47   26     59785.94                236.50   \n",
       "3                     74.15   29     54806.18                245.89   \n",
       "4                     68.37   35     73889.99                225.58   \n",
       "\n",
       "                           Ad Topic Line            City  Male     Country  \\\n",
       "0     Cloned 5thgeneration orchestration     Wrightburgh     0     Tunisia   \n",
       "1     Monitored national standardization       West Jodi     1       Nauru   \n",
       "2       Organic bottom-line service-desk        Davidton     0  San Marino   \n",
       "3  Triple-buffered reciprocal time-frame  West Terrifurt     1       Italy   \n",
       "4          Robust logistical utilization    South Manuel     0     Iceland   \n",
       "\n",
       "             Timestamp  Clicked on Ad  \n",
       "0  2016-03-27 00:53:11              0  \n",
       "1  2016-04-04 01:39:02              0  \n",
       "2  2016-03-13 20:35:42              0  \n",
       "3  2016-01-10 02:31:19              0  \n",
       "4  2016-06-03 03:36:18              0  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ad_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll have the freedom here to choose columns that you want to train on!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Split the data into training set and testing set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# taking required columns\n",
    "new_ad_data = ad_data[[\"Daily Time Spent on Site\",\"Age\",\"Area Income\",\"Daily Internet Usage\",\"Male\"]]\n",
    "new_target = ad_data[\"Clicked on Ad\"]\n",
    "# normalising the data\n",
    "# new_ad_data[\"Daily Time Spent on Site\"] = (new_ad_data[\"Daily Time Spent on Site\"]-np.mean(new_ad_data[\"Daily Time Spent on Site\"]))/ np.std(new_ad_data[\"Daily Time Spent on Site\"])\n",
    "# new_ad_data[\"Age\"] = (new_ad_data[\"Age\"]-np.mean(new_ad_data[\"Age\"]))/ np.std(new_ad_data[\"Age\"])\n",
    "# new_ad_data[\"Area Income\"] = (new_ad_data[\"Area Income\"]-np.mean(new_ad_data[\"Area Income\"]))/ np.std(new_ad_data[\"Area Income\"])\n",
    "# new_ad_data[\"Daily Internet Usage\"] = (new_ad_data[\"Daily Internet Usage\"]-np.mean(new_ad_data[\"Daily Internet Usage\"]))/ np.std(new_ad_data[\"Daily Internet Usage\"])\n",
    "\n",
    "[X_train,X_test,y_train,y_test] = train_test_split(new_ad_data,new_target,random_state=0,train_size=0.8)\n",
    "# [x_new_train,x_new_test,y_new_train,y_new_test] = train_test_split(new_ad_norm_data,new_target,random_state=0,train_size=0.8)\n",
    "\n",
    "\n",
    "# print(x_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Train and fit a logistic regression model on the training set.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=LogisticRegression()\n",
    "clf = model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predictions and Evaluations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict the values for test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set score: 0.892\n",
      "Test set score: 0.915\n"
     ]
    }
   ],
   "source": [
    "print(\"Train set score: {:.3f}\".format(clf.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.3f}\".format(clf.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set score: 0.971\n",
      "Test set score: 0.980\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  \n",
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n",
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  after removing the cwd from sys.path.\n",
      "/home/meenu/.local/lib/python3.5/site-packages/ipykernel_launcher.py:5: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  \"\"\"\n"
     ]
    }
   ],
   "source": [
    "# normalising the data\n",
    "new_ad_data[\"Daily Time Spent on Site\"] = (new_ad_data[\"Daily Time Spent on Site\"]-np.mean(new_ad_data[\"Daily Time Spent on Site\"]))/ np.std(new_ad_data[\"Daily Time Spent on Site\"])\n",
    "new_ad_data[\"Age\"] = (new_ad_data[\"Age\"]-np.mean(new_ad_data[\"Age\"]))/ np.std(new_ad_data[\"Age\"])\n",
    "new_ad_data[\"Area Income\"] = (new_ad_data[\"Area Income\"]-np.mean(new_ad_data[\"Area Income\"]))/ np.std(new_ad_data[\"Area Income\"])\n",
    "new_ad_data[\"Daily Internet Usage\"] = (new_ad_data[\"Daily Internet Usage\"]-np.mean(new_ad_data[\"Daily Internet Usage\"]))/ np.std(new_ad_data[\"Daily Internet Usage\"])\n",
    "\n",
    "[X_train,X_test,y_train,y_test] = train_test_split(new_ad_data,new_target,random_state=0,train_size=0.8)\n",
    "\n",
    "model=LogisticRegression()\n",
    "clf = model.fit(X_train, y_train)\n",
    "print(\"Train set score: {:.3f}\".format(clf.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.3f}\".format(clf.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multi-class Logistic Regression "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One vs. All Classifier\n",
    "<br>\n",
    "Use multiclass option in Logistic Regression function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Wine</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Wine  Alcohol  Malic.acid   Ash   Acl   Mg  Phenols  Flavanoids  \\\n",
       "0     1    14.23        1.71  2.43  15.6  127     2.80        3.06   \n",
       "1     1    13.20        1.78  2.14  11.2  100     2.65        2.76   \n",
       "2     1    13.16        2.36  2.67  18.6  101     2.80        3.24   \n",
       "3     1    14.37        1.95  2.50  16.8  113     3.85        3.49   \n",
       "4     1    13.24        2.59  2.87  21.0  118     2.80        2.69   \n",
       "\n",
       "   Nonflavanoid.phenols  Proanth  Color.int   Hue    OD  Proline  \n",
       "0                  0.28     2.29       5.64  1.04  3.92     1065  \n",
       "1                  0.26     1.28       4.38  1.05  3.40     1050  \n",
       "2                  0.30     2.81       5.68  1.03  3.17     1185  \n",
       "3                  0.24     2.18       7.80  0.86  3.45     1480  \n",
       "4                  0.39     1.82       4.32  1.04  2.93      735  "
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df=pd.read_csv('wine.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the dataset into train and test sets\n",
    "<br>\n",
    "Use 60-40 train-test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "target = df[\"Wine\"]\n",
    "wine_data = df.drop(columns=\"Wine\")\n",
    "[X_train,X_test,y_train,y_test] = train_test_split(wine_data,target,random_state=0,train_size=0.6)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply Multiclass Logistic Regression, Knn and Multinominal Naive Bayes classifiers and get the accuracy scores.\n",
    "<br>\n",
    "Use different values of k to get the best possible accuracy in case of Knn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNN\n",
      "Train set score: 1.000\n",
      "Test set score: 0.750\n",
      "Train set score: 0.887\n",
      "Test set score: 0.667\n",
      "Train set score: 0.811\n",
      "Test set score: 0.708\n",
      "Train set score: 0.764\n",
      "Test set score: 0.736\n",
      "Train set score: 0.764\n",
      "Test set score: 0.694\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VdW1wPHfykRIgAAhECBAAoQxTBoGARlkCmpxap1b22fFPofW2qr42mqrz5a21qqtw0OLrbVqK86KTAriADKJkDAlDEIggRDmIWRa749zQi4I5BJucu7NXd/P53649wz3rGiyzjn7rL23qCrGGGPCQ4TXARhjjKk/lvSNMSaMWNI3xpgwYknfGGPCiCV9Y4wJI5b0jTEmjFjSN8aYMGJJ3xhjwoglfWOMCSNRXgdwslatWmlqaqrXYRhjTEhZvnz5blVNqmm7oEv6qampLFu2zOswjDEmpIjI1/5sZ807xhgTRizpG2NMGLGkb4wxYSTo2vSNMaY2ysrKyM/Pp6SkxOtQ6lRsbCwpKSlER0fXan9L+saYBiE/P5+mTZuSmpqKiHgdTp1QVYqLi8nPzyctLa1W31Fj846ITBeRXSKSfZr1IiJPikieiKwSkfN81t0kIrnu66ZaRWiMMX4oKSkhMTGxwSZ8ABEhMTHxnO5m/GnT/zuQdYb1E4F09zUZeMYNriXwIDAYGAQ8KCItah2pMcbUoCEn/Crn+jPWmPRVdSGw5wybXAa8qI7FQHMRaQtMAOaq6h5V3QvM5cwnj3NytLSCqR+sY9ueI3V1CGOMCXmBqN5pD2zz+ZzvLjvd8m8QkckiskxElhUVFdUqiL1HSvnnoi38z5ursXl/jTH1bd++fTz99NNnvd/FF1/Mvn376iCiUwuKkk1VnaaqmaqamZRUYy/iU2rXvDFTJvbgk9zdzFieH+AIjTHmzE6X9MvLy8+438yZM2nevHldhfUNgUj624EOPp9T3GWnW15nbhjciYGpLXj4vTXsOtiwy7aMMcFlypQpbNy4kf79+zNw4EAuvPBCJk2aRK9evQC4/PLLOf/88+nduzfTpk07vl9qaiq7d+9my5Yt9OzZk1tuuYXevXszfvx4jh49GvA4A1Gy+Q5wh4i8ivPQdr+qFojIbOC3Pg9vxwP3B+B4pxURIUy9qi8Tn/iEX7+Tw9M3nF+XhzPGBKnfvJvDmh0HAvqdvdo148Fv9T7t+qlTp5Kdnc3KlStZsGABl1xyCdnZ2cdLK6dPn07Lli05evQoAwcO5KqrriIxMfGE78jNzeWVV17hueee4+qrr+b111/nxhtvDOjP4U/J5ivAIqC7iOSLyM0i8iMR+ZG7yUxgE5AHPAfcBqCqe4CHgaXu6yF3WZ3qktSEu8amM3N1IbOyC+r6cMYYc0qDBg06oZb+ySefpF+/fgwZMoRt27aRm5v7jX3S0tLo378/AOeffz5btmwJeFw1Xumr6nU1rFfg9tOsmw5Mr11otXfLhZ1576sCfvV2Dhd0bkVCXO16rhljQtOZrsjrS3x8/PH3CxYsYN68eSxatIi4uDhGjRp1ylr7Ro0aHX8fGRlZJ807QfEgN9CiIyP4w7f7sudwKY/MXON1OMaYMNC0aVMOHjx4ynX79++nRYsWxMXFsW7dOhYvXlzP0VVrsMMwZLRPYPKIzjyzYCOT+rVneHorr0MyxjRgiYmJDBs2jIyMDBo3bkybNm2Or8vKyuLZZ5+lZ8+edO/enSFDhngWpwRbTXtmZqYGahKVkrIKLn7iE8oqK5l91wjiYhrsOc6YsLd27Vp69uzpdRj14lQ/q4gsV9XMmvZtkM07VWKjI5l6VV+27TnKn+Zs8DocY4zxXINO+gCD0lpy45COTP9sMyu27vU6HGOM8VSDT/oA92X1ILlZLPfNWMWx8gqvwzHGGM+ERdJvGhvNI1dkkLvrEE/P3+h1OMYY45mwSPoAF/Vow+X92/H0gjzWF566rMoYYxq6sEn6AA98qzdNY6O59/VVVFQGV9WSMcbUh7BK+i3jY3jwW734ats+Xvhss9fhGGMakNoOrQzw+OOPc+RI/cwFElZJH2BSv3aM6dGaR+esZ2uxTbhijAkMS/pBSkT43ysyiIqIYMobq2zCFWNMQPgOrXzPPffwxz/+kYEDB9K3b18efPBBAA4fPswll1xCv379yMjI4N///jdPPvkkO3bsYPTo0YwePbrO4wzLLqptExpz/8U9+MWb2by2LJ+rB3aoeSdjTOj4YAoUrg7sdyb3gYlTT7vad2jlOXPmMGPGDJYsWYKqMmnSJBYuXEhRURHt2rXj/fffB5wxeRISEnjssceYP38+rVrV/XAxYXelX+W6gR0ZnNaSh99fw84DNuGKMSZw5syZw5w5cxgwYADnnXce69atIzc3lz59+jB37lzuu+8+PvnkExISEuo9trC80ofqCVeyHl/Ir97K5v++e/45zzJvjAkSZ7girw+qyv3338+tt976jXUrVqxg5syZ/PKXv2TMmDE88MAD9Rpb2F7pA6S1iuen47oxZ81OPsgu9DocY0wI8x1aecKECUyfPp1Dhw4BsH37dnbt2sWOHTuIi4vjxhtv5J577mHFihXf2Leuhe2VfpUfDk/jvVU7eODtbIZ2SaR5XIzXIRljQpDv0MoTJ07k+uuv54ILLgCgSZMmvPTSS+Tl5XHPPfcQERFBdHQ0zzzzDACTJ08mKyuLdu3aMX/+/DqNs0EPreyvNTsOMOmvn3L5gPY8+p1+9XpsY0xg2NDKNrSy33q1a8aPRnZhxvJ8Fm4o8jocY4ypM5b0XXdc1JXOSfHc/8ZqDh8r9zocY4ypE34lfRHJEpH1IpInIlNOsb6TiHwoIqtEZIGIpPisqxCRle7rnUAGH0ix0ZH84aq+7Nh/lD/OXu91OMaYWgi25uq6cK4/Y41JX0QigaeAiUAv4DoR6XXSZo8CL6pqX+Ah4Hc+646qan/3Nemcoq1jmakt+d6QTvxj0RaWf20TrhgTSmJjYykuLm7QiV9VKS4uJjY2ttbf4U/1ziAgT1U3AYjIq8BlwBqfbXoBd7vv5wNv1Toij92T1YN5a3dx3+ureP/Hw2kUFel1SMYYP6SkpJCfn09RUcN+LhcbG0tKSkrNG56GP0m/PbDN53M+MPikbb4CrgSeAK4AmopIoqoWA7EisgwoB6aqalCfEJo0iuKRKzL4/gtLeeqjPO4e393rkIwxfoiOjiYtLc3rMIJeoB7k/hwYKSJfAiOB7UDVvISd3DKi64HHRaTLyTuLyGQRWSYiy4LhLD2qe2uuHNCepxdsZG3BAa/DMcaYgPEn6W8HfEckS3GXHaeqO1T1SlUdAPzCXbbP/Xe7++8mYAEw4OQDqOo0Vc1U1cykpKTa/BwB96tLe5HQOJr7Xl9FeUWl1+EYY0xA+JP0lwLpIpImIjHAtcAJVTgi0kpEqr7rfmC6u7yFiDSq2gYYxonPAoJWi/gYfnNZb1bl72e6TbhijGkgakz6qloO3AHMBtYC/1HVHBF5SESqqnFGAetFZAPQBnjEXd4TWCYiX+E84J2qqiGR9AEu6dOWcb3a8NjcDWzZfdjrcIwx5pzZMAw12HmghLGPfUzvds145ZYhNhKnMSYo2TAMAdKmWSz/c3FPFm/aw6tLt9W8gzHGBDFL+n64dmAHLuicyG/fX0vhfptwxRgTuizp+0FE+N2VfSirrOSXb2U36B5/xpiGzZK+n1JbxfOzcd2Zt3Yn768u8DocY4ypFUv6Z+EHw1Lpl5LAg2/nsPdwqdfhGGPMWbOkfxaiIiOYelVf9h8t4+H3Qqby1BhjjrOkf5Z6tm3GbaO68MaX25m/fpfX4RhjzFmxpF8Lt1/Ula6tm/CLN1ZzyCZcMcaEEEv6tdAoKpLfX9WXggMl/HHWOq/DMcYYv1nSr6XzO7Xg+0NTeXHx1yzdssfrcIwxxi+W9M/Bz8d3p11CY+57fRUlZRU172CMMR6zpH8O4htF8bsr+7Cp6DB/+SjX63CMMaZGlvTP0YhuSXz7/BSe/XgTOTv2ex2OMcackSX9APjlJT1pERdjE64YY4KeJf0AaB4Xw8OX9SZ7+wGe/9QmXDHGBC9L+gEysU9bJvRuw5/nbmBT0SGvwzHGmFOypB9AD1+WQUxUBFPeWE1lpY3EaYwJPpb0A6h1s1h+dUkvlmzew8tLtnodjjHGfIMl/QD7TmYKw7omMvWDdRTsP+p1OMYYcwJL+gEmIvzuir5UVCq/eNMmXDHGBBdL+nWgY2IcP5/QnY/W7eKdr3Z4HY4xxhznV9IXkSwRWS8ieSIy5RTrO4nIhyKySkQWiEiKz7qbRCTXfd0UyOCD2feHptK/Q3N+8+4aig8d8zocY4wB/Ej6IhIJPAVMBHoB14lIr5M2exR4UVX7Ag8Bv3P3bQk8CAwGBgEPikiLwIUfvCIjhN9f1ZeDJWU8ZBOuGGOChD9X+oOAPFXdpKqlwKvAZSdt0wv4yH0/32f9BGCuqu5R1b3AXCDr3MMODd2Tm3L76K68vXIHH63b6XU4xhjjV9JvD2zz+ZzvLvP1FXCl+/4KoKmIJPq5b4N226iudG/TlF+8mc3BkjKvwzHGhLlAPcj9OTBSRL4ERgLbAb/HGhaRySKyTESWFRUVBSik4BATFcHvv92XnQdK+L1NuGKM8Zg/SX870MHnc4q77DhV3aGqV6rqAOAX7rJ9/uzrbjtNVTNVNTMpKeksf4Tg179Dc34wLI2XFm/li03FXodjjAlj/iT9pUC6iKSJSAxwLfCO7wYi0kpEqr7rfmC6+342MF5EWrgPcMe7y8LOz8Z3o0PLxkx5Y7VNuGKM8UyNSV9Vy4E7cJL1WuA/qpojIg+JyCR3s1HAehHZALQBHnH33QM8jHPiWAo85C4LO3ExUUy9si+bdx/miQ9twhVjjDck2HqMZmZm6rJly7wOo87cN2MVM1bk8/btw8hon+B1OMaYBkJElqtqZk3bWY/cevY/l/QkMT6Ge2esoswmXDHG1DNL+vUsoXE0D12WwZqCA/z1ozwbm8cYU68s6XsgKyOZS/u25YkPc7nqmc/5PG+31yEZY8KEJX2P/Pma/vz2ij4U7C/h+ue/4Lppi1m2JSyfcRtj6pE9yPVYSVkFryzZylPzN7L70DFGdkviZ+O70TeludehGWNCiL8Pci3pB4kjpeW8uOhrnv14I/uOlDG+VxvuHt+NHsnNvA7NGBMCLOmHqIMlZbzw2RaeW7iJQ6XlXNKnLXeN7UbX1k28Ds0YE8Qs6Ye4/UfKeO6TTUz/bDMlZRVcMSCFn4xJp2NinNehGWOCkCX9BqL40DGe/XgjLy76mopK5TuZHbjzoq60a97Y69CMMUHEkn4Ds/NACU/Nz+OVJVsRhOsHd+S20V1o3TTW69CMMUHAkn4Dlb/3CH/9KI/XlucTHSncdEEqt47sQsv4GK9DM8Z4yJJ+A7fFHbjtrZXbiYuO5Obhadx8YWcSGkd7HZoxxgOW9MNE7s6DPD4vl/dXF9AsNorJIzrz/WFpNGkU5XVoxph6ZEk/zOTs2M+f525g3tpdtIyP4b9HduHGIZ1oHBPpdWjGmHpgST9Mfbl1L4/N3cAnubtJatqIO0Z35dpBHWgUZcnfmIbMkn6Y+2JTMX+as4ElW/bQLiGWH49J56rzU4iOtOGWjGmILOkbVJVP83bzpzkbWLltH50S4/jJmHQu69+eyAjxOjxjTADZJCoGEeHC9CTevG0of7spk/iYKO7+z1dMeHwh768qoLIyuE74xpi6Z0k/DIgIY3q24b07h/P0DechwO0vr+CSv3zK3DU7bSIXY8KIJf0wEhEhXNynLbPuGsHj1/TnaGk5t7y4jMuf+oyFG4os+RsTBizph6HICOHyAe2Zd/dI/nBVX3YfKuV705dwzf8t5otNxV6HZ4ypQ34lfRHJEpH1IpInIlNOsb6jiMwXkS9FZJWIXOwuTxWRoyKy0n09G+gfwNReVGQEVw/swEc/H8nDl/VmS/Fhrpm2mBuf/4IVW/d6HZ4xpg7UWL0jIpHABmAckA8sBa5T1TU+20wDvlTVZ0SkFzBTVVNFJBV4T1Uz/A3Iqne8U1JWwUuLv+aZBRspPlzKmB6t+em4bmS0T/A6NGNMDQJZvTMIyFPVTapaCrwKXHbSNgpUTfGUAOw4m2BNcIiNjuSHF3Zm4b2juWdCd5Zu2cOlf/mU/35pORt2HvQ6PGNMAPiT9NsD23w+57vLfP0auFFE8oGZwJ0+69LcZp+PReTCUx1ARCaLyDIRWVZUVOR/9KZOxDeK4vbRXfnkvov48Zh0PsndzYTHF3LXq1+yefdhr8MzxpyDQD3IvQ74u6qmABcD/xSRCKAA6KiqA4C7gZdF5BuTvqrqNFXNVNXMpKSkAIVkzlVC42juHteNT+4dza0jujArp5Cxj33MvTO+YtueI16HZ4ypBX+S/nagg8/nFHeZr5uB/wCo6iIgFmilqsdUtdhdvhzYCHQ716BN/WoRH8OUiT1YeO9ovndBJ976cgcX/WkBv3orm8L9JV6HZ4w5C/4k/aVAuoikiUgMcC3wzknbbAXGAIhIT5ykXyQiSe6DYESkM5AObApU8KZ+tW4ay4Pf6s3H947i6swOvLJkKyP+OJ+H31vD7kPHvA7PGOMHv8becUswHwcigemq+oiIPAQsU9V33Iqd54AmOA9171XVOSJyFfAQUAZUAg+q6rtnOpZV74SObXuO8MSHubyxIp/Y6Ei+PzSVySM60zzOZvEypr7ZgGum3mwsOsQT83J5d9UOmsREcfOFadw8PI2msTaLlzH1xZK+qXfrCg/w57kbmJ2zk+Zx0dw6ogs3De1EXIzN4mVMXbOkbzyzOn8/j81dz/z1RbRqEsN/j+rKDYM7EhttE7kYU1cs6RvPLf96D3+as4HPNxaT3CyWOy7qytWZHYiJsiGfjAk0S/omaHy+0ZnIZfnXe0lp0Zgfj0nnygHtibJZvIwJGJtExQSNoV1aMeNHF/D3HwykRVwM985Yxbg/L+TtldttIhdj6pklfVMvRIRR3Vvzzh3DmPbd82kUFcFPXl1J1hMLmZVdYGP5G1NPLOmbeiUijO+dzMwfX8hfrhtAeaXyo5dW8K2/fsr8dbss+RtTxyzpG09ERAjf6teOOXeN4NHv9GP/0TJ+8PelXPnM53yWt9uSvzF1xB7kmqBQVlHJa8vy+ctHuRTsL2FI55b8bHx3Bqa29Do0Y0KCVe+YkFRSVsErS7by1PyN7D50jJHdkvjZ+G70TWnudWjGBDVL+iakHSkt55+LvubZjzey90gZ43q14e5x3ejZ9hsjcxtjsKRvGoiDJWW88NkWnlu4iYPHyrm0b1vuGtuNrq2beB2aMUHFkr5pUPYfKeO5TzYx/bPNlJRVcPmA9vxkTDqdEuO9Ds2YoGBJ3zRIxYeO8ezHG3lx0ddUVCrfyUzhjovSad+8sdehGeMpS/qmQdt5oISn5+fx8pKtCML1gzty26gutG4W63VoxnjCkr4JC/l7j/DXj/J4bXk+0ZHCTRekcuvILrSMt4lcTHixpG/Cypbdh3nyw1zeXLmduOhIrh/ckUv6tqNfSgIi4nV4xtQ5S/omLOXuPMjj83KZnVNIeaXSNiGWCb2TycpIZmBqSyIj7ARgGiZL+ias7TtSyodrd/FBdiELc4soLa8kMT6G8b3bMKF3MkO7tLJx/U2DYknfGNfhY+UsWF/EB9kFzF+3i8OlFTSNjWJsT+cEMLJbEo1jbFYvE9oCmvRFJAt4AogEnlfVqSet7wj8A2jubjNFVWe66+4HbgYqgB+r6uwzHcuSvqlLJWUVfJq7m1k5hcxds5P9R8toHB3JqO5JZGUkc1GP1jahuwlJAUv6IhIJbADGAfnAUuA6VV3js8004EtVfUZEegEzVTXVff8KMAhoB8wDuqlqxemOZ0nf1Jeyikq+2LSHWTkFzM7ZSdHBY8RERjCsayITM9oytlcbqwIyIcPfpB/lx3cNAvJUdZP7xa8ClwFrfLZRoGpQlARgh/v+MuBVVT0GbBaRPPf7Fvn1UxhTh6IjIxie3orh6a14aFIGK7buZVZ2IbNyCpn/+ioi3oDBaYlM7JPM+F7JJCdYHwAT+vxJ+u2BbT6f84HBJ23za2COiNwJxANjffZdfNK+7WsVqTF1KCJCyExtSWZqS35xSU9ydhw4fgJ44O0cHng7hwEdmzMxI5ms3m3pmBjndcjG1Io/Sd8f1wF/V9U/icgFwD9FJMPfnUVkMjAZoGPHjgEKyZjaEREy2ieQ0T6Bn0/oTt6ug8dPAL+duY7fzlxHz7bNnBNARjLprZtYXwATMvxJ+tuBDj6fU9xlvm4GsgBUdZGIxAKt/NwXVZ0GTAOnTd/f4I2pD11bN+WOi5pyx0XpbNtzhNk5hczKLuTP8zbw2NwNdG4VT5Z7AujT3jqDmeDmz4PcKJwHuWNwEvZS4HpVzfHZ5gPg36r6dxHpCXyI04zTC3iZ6ge5HwLp9iDXNAS7DpQwe81OZmcXsmhTMRWVSvvmjY93Bju/UwvrDGbqTaBLNi8GHscpx5yuqo+IyEPAMlV9x63SeQ5ogvNQ915VnePu+wvgv4By4C5V/eBMx7Kkb0LR3sOlzFu7k9k5hSzM3U1peSWtmjRifO82ZPVO5oIuiURHWmcwU3esc5YxHjl0rJz563Y5VUDrdnGktIJmsVGM7eWcAEZ0SyI22jqDmcCypG9MECgpq+CT3N3Myi5k3lqnM1hcTCSju7dmgtsZrEmjQNVTmHAWyDp9Y0wtxUZHMq5XG8b1akNZRSWLNxUzK7uQ2Tk7eX91ATGREVyY3ooJGcmM69mGFtYZzNQxu9I3xgMVlVrdGSy7kO37jhIZIQzp3JKs3slM6J1sE8KYs2LNO8aECFUle/sBPsguYFZ2IZt2H0YEzuvYgiy3EqhDS+sMZs7Mkr4xIUhVydt1iA/cO4A1BQcA6N2uGVm9k5nYJ5murZt6HKUJRpb0jWkAthYfYVaOcwewYus+ALokOZ3BJma0pXe7ZtYZzACW9I1pcAr3lzBnjXMH8MXmPcc7gzkngGTO69iCCOsMFrYs6RvTgO05XMq8NTuZlVPIp7m7Ka2oJKlpI8b3asPEjLYM7tzSOoOFGUv6xoSJgyVlfLRuF7NzCpm/roijZRUkNI5mbM82TMxIZnh6K+sMFgYs6RsThkrKKvh4QxGzswuZu3YnB0vKiY+JZHSP1mRlJDOqu3UGa6isc5YxYSg2OpIJbp1/aXkli9zOYHPXFPLeqgJioiIYke5MDTm2Z2uax1lnsHBjV/rGhIGKSmXZlj3MyilkdnYhO/aXEBUhXNAlkQm9kxnfuw2tm1pnsFBmzTvGmFNSVVbl72eWOy/AZrczWGanFsfvEqwzWOixpG+MqZGqsmHnIWZlF/JBdgHrCg8C0Kd9AlkZzgmga+smHkdp/GFJ3xhz1rbsPszsnEI+yC5k5TanM1h66ybHTwDWGSx4WdI3xpyTgv1HmZOzkw+yC1iyeQ+VCh1aNqZ/hxZYH7C60SkxnrvHdavVvla9Y4w5J20TGnPT0FRuGppK8aFjzFu7kw+yC1mdv8/r0Bqs0vLKOj+GJX1jTI0SmzTimoEduWZgR69DMefI+mkbY0wYsaRvjDFhxJK+McaEEb+Svohkich6EckTkSmnWP9nEVnpvjaIyD6fdRU+694JZPDGGGPOTo0PckUkEngKGAfkA0tF5B1VXVO1jar+1Gf7O4EBPl9xVFX7By5kY4wxteXPlf4gIE9VN6lqKfAqcNkZtr8OeCUQwZ21DXOgrMSTQxtjTCjwJ+m3B7b5fM53l32DiHQC0oCPfBbHisgyEVksIpefZr/J7jbLioqK/Az9JEUb4OWr4a0fQWXd17oaY0woCvSD3GuBGapa4bOsk9tL7HrgcRHpcvJOqjpNVTNVNTMpKal2R07qBuMegpw3Ye6vavcdxhjTwPnTOWs70MHnc4q77FSuBW73XaCq291/N4nIApz2/o1nHak/ht4J+7fBor9C844w+NY6OYwxxoQqf670lwLpIpImIjE4if0bVTgi0gNoASzyWdZCRBq571sBw4A1J+8bMCKQNRV6XAof3Adr362zQxljTCiqMemrajlwBzAbWAv8R1VzROQhEZnks+m1wKt64ghuPYFlIvIVMB+Y6lv1UyciIuGq5yElE17/IWxbUqeHM8aYUNJwR9k8vBv+Ng6O7oOb50Krruf+ncYYE6T8HWWz4fbIjW8FN8xwmnz+dRUcqmVVkDHGNCANN+kDJHaB6/8DB3fCK9dA6WGvIzImtKk6LxOyGv7QyimZ8O3p8O8bYMbNcM1LENnwf2xjzllFORTnQsEqKKx6rQaJgC4XQddx0HWMc1dtQkZ4ZL8eF8PEP8DMn8MH98Ilf3KafYwxjtLDsDPHSewFbnLftQbK3R7ukY2gTS/o+S0oPwZ5H8Lq1wCBdv2dE0D6OGh/vlNMYYJWeCR9gEG3ODX8nz0BzTvA8J/WvI8xDdGhohOv3AtWQXEe4DbbxDaH5D4w8IfOv8l9oVU6REZXf0dlJRSshLx5kDsXPnkUFv7B2bfLRc4JoOtYaNLakx/RnF7Drd45lcpKeOMWyJ4BVz4Pfb9TN8cxJhhUVsK+LdVX7lVJ/mBB9TYJHd3E3gfa9nX+Tehw9nfCR/bApvmQO885ERze5Sxv289J/l3HQcpAa1qtQzYx+umUH4OXroKti+G7b0DaiLo7ljH1pbwUitZWX7kXrnZepQed9RIJSd2rr9yrEn1cy8DHUlnpnGDy3BPAtiWgFRCbAJ1HuyeBsdCsbeCPHcYs6Z/J0X0wfQIcKID/muW0VRoTKkr2Q2H2ic0zReugssxZHx0PyRknJvjWPSG6sTfxHt3n3AXkzXPuBA4VOsvb9HEeBKePgw6DT2w+MmfNkn5N9m2D58dCRBT8cJ5ddZjgo+o0xRy/cv/K+Xfvlupt4pOcxF7VNJPcD1qmBe/DVFXYmV19Ati2GCrLoVEz6Dyyuiko4ZQD+ZozsKTvj4JV8MJEaJEGP5gJsc3q57jGnKyywnmYWrgaCr6qboM/Uly9TcvO1Vfubfs5/zZN9i7mQCjZD5s+rm44HMcTAAAPD0lEQVQKOuCO5di6l3MCSB8HHYZAVIy3cYYAS/r+yvvQGYc/9UK44TW7xTR1r/QI7FpbfeVesMoplyw/6qyPjHGaY6qu3JP7OM01jZp6G3ddU3X+u+TNdU4AXy9ymqximkDaSEh37wKad6j5u8KQJf2z8eW/4O3boN/1cPnTVsNvAufInhOv3AtXw+4NoO5EP40STqycSe4LrbrZlS3AsYOweaFTEpo3zym5BmjVvboktNNQiGrkbZxBwt+kb/VTAANucH6hFvzOuYoY/T9eR2RCjSrs+/qk6plV1c0VAM3aO0m956TqJN+8k11knE6jptDjEuel6pwsc+c6dwJLpjnzZkTHORV4VU1BLVK9jjroWdKvMvI+J/F//HtISIHzvud1RCZYVZRB0foTq2cKV8Ox/c56iXCu1jsN9SmP7Avxid7GHcpEnJLTpO4w9A6nB/HmT5wTQO5c2DDL2S6xq9s7eCx0Gg7Rsd7GHYSsecdXRRm8fA1sWuAM1JY+1ps4TPAoOeAOT+BTPbNrLVSUOuujGkOb3idWz7TuCTFx3sYdTlSdh+BVvYO3fAoVx5z/N6nDq5uCEr8xU2uDYm36tXXsILxwMRRvdCp62vX3LhZTvw4WfnNwsT2bqtfHJX6zeiaxa/CWR4ar0iPw9WfVTUFV/w9bdq4uCU0d3uBOzJb0z8XBQqeGv6LUmYClRSdv4zGBVVkJezaeOLhY4erqoQPAaRv2rZ5p2xeatrX291BUvNGp0sub6zQJlR91BpBLHV79LCCxa8j/v7Wkf652rYPp46FJMtw8Gxq38DoiUxtlJc5okb7t7ztzoMydWyEiCpJ6nlg9k5zhDBlgGp6yo85dQN6Hzp1Aca6zvHmn6magtBEQE+9tnLVgST8QtnwG/7zcGSjqxjfsoVAo2LfVadvduthJ8kXrnXFfAGKafnNwsaQeVvIXzvZucZuBPoTNH0PZEaefRKeh1U1BSd1D4i7Akn6grJ4Br98Mva+Aq6ZDRMOebCzklB+Drz+vfoi3e72zvEkbt929b3WSb55q///M6fn+LuXNc8YzAmfU0apB4jqPDNpOcgFN+iKSBTwBRALPq+rUk9b/GRjtfowDWqtqc3fdTcAv3XX/q6r/ONOxgi7pgzMG/9wHYOidMP5/vY7G7Nlc/Ye5eaHP1dkw9xZ9nDP+ewhcnZkgVnXXmPehU9FXeggioqHjkOqmoNa9gub3LGBJX0QigQ3AOCAfWApcp6prTrP9ncAAVf0vEWkJLAMycWZoWA6cr6p7T3e8oEz6qs6MW0umwcQ/wuDJXkcUXqraYXPnOQ/jivOc5S1Sq2dsSh0eku2wJkSUlzqDw1U1Be3KcZY3a++MFNp1nHMX4OGzoED2yB0E5KnqJveLXwUuA06Z9IHrgAfd9xOAuaq6x913LpAFvOLHcYOHCGRNhQM7nOTfrK0zbZypO8UbT6y7Lj8KUbFOch94S3XddZBcZZkGLirGecCbNgLGPwz7t7t3AXMh5y1Y8aJTFNBhcHVFUJuMoPz99Cfptwe2+XzOBwafakMR6QSkAR+dYd/QHDM1IhKufA5enASv/xBuehc6DPI6qoaj9IiT3Kt6WO7d7Cxv2QXOv8n5Q0od7t2Y8Mb4Smjv/F6ef5PTqXPbEvd3dx58+Bvn1STZPQGMhc6jgqYCMNDDMFwLzFCtKpfwj4hMBiYDdOzYMcAhBVBMHFz3KvxtnNNz94fzGnwvvzpT1YuyqgPNls+qe1GmjYAhtzl/LC07ex2pMWcWGQ2pw5zX2F87kzNtdEtC174LK19yZi5LGVg9UmhyX8+KCvxp078A+LWqTnA/3w+gqr87xbZfArer6ufu5+uAUap6q/v5/4AFqnra5p2gbNM/WfFGJ/E3auZ03mqS5HVEoaH0sPPgtarZZt/XzvJW3aqrIzoNs9JY03BUlEP+0uqmoIKvnOXxrd1nAWOdieQDMG1lIB/kRuE8yB0DbMd5kHu9quactF0PYBaQpu6Xug9ylwPnuZutwHmQu+d0xwuJpA+Qvwz+fqkz1eJN7zW4Lt0BoerUyR8fH/1zp5dzdLw7S5L7S28jI5pwcXAnbPzI+ZvY+BEc3esM0Nc+s/pZQPvzav6eUwh0yebFwOM4JZvTVfUREXkIWKaq77jb/BqIVdUpJ+37X0DVWMWPqOoLZzpWyCR9gHXvw79vhG5ZcM1LNgYLOGMX+c6EVDUGelKP6l/qjhdYhyhjKitg+/LqO98dXzpjfU1eUKuvs85Z9WXJczDz5zDwh3Dxo0H5tL5OqTrDHFT94m5dXD3bUedR1c02NtuRMWd2eLcz7ldyRq12t0lU6sugW5yr2c+ecHruDb/L64jq3vF5Td2a5ePzmvaGC25zHlR1GGyzPxlzNuJbOa86Zkk/EMb8Gvbnw7wHnQlY+nzb64gCSxV2ZldPW7ftC6gsdx5kdx4Fo6ZAlzFOGZsxJqhZ0g+EiAi4/BnnIc2bP3LGfUm70Ouozs3RvU7X81y3bf5QobM8uQ8M/bHTZNNhkE0kb0yIsaQfKFGN4NqXYHoWvHqDMxxz655eR+W/ykpn+OGqDib5S53RKWMTnJKyruOcapumyV5Haow5B5b0A6lxC7jhNXh+HLz0bafzVrO2Xkd1ekf2uOVj7qBSVZOItO0PF97tXM23z4RI+zUxpqGwv+ZAa97RSfwvTIR/fceZcjG2mddROSoroeDL6oHLti8HrXROVl3cmvmuY6BJa68jNcbUEUv6daFtX7j6H/Cvq+E/33NOAl61fR8udrqEV13NH9kNiNMBZMQ9TrNN+/Osj4ExYcKSfl3pOhYmPQlv3w7v/gQue6p+avgrK2D7iupu39tXAOpM6t1ljNM5qstF9VIaZowJPpb069KAG51SzgW/c2r4R99fN8c5tMtpm8+t6tq9p7pr96j7nUGe2g6wWaOMMZb069zI+5zOWx9PdWr4z/vuuX9nRTlsX1bdC7ZgpbM8Pgm6TQjoIE7GmIbFkn5dE4FLH3eGW333J9C0rXPlfbYOFjpt8nlzYeN8KNnnXM2nDIKLfun5cK3GmNBgSb8+REY7D3ZfmAiv3eRU9LTtd+Z9KsqcWvmq8eYLVzvLmyRDj0uDbmIGY0xosKRfXxo1hetfc8bh/9d3nBr+5idNGHNgR3WTzaaP4dh+Z/KFjkNgzINBPQWbMSY0WNKvT83awg0zYPp4p/PW99+HonXVvWCrJltu2g56XxYUky0bYxoWS/r1rXUPuPZl+OcV8Gg6oBAR7VzNj3vISfSte9rVvDGmTljS90LqcLj6RefBbOdRztV8o6ZeR2WMCQOW9L3SfaLzMsaYemT1fcYYE0Ys6RtjTBixpG+MMWHEkr4xxoQRv5K+iGSJyHoRyRORKafZ5moRWSMiOSLyss/yChFZ6b7eCVTgxhhjzl6N1TsiEgk8BYwD8oGlIvKOqq7x2SYduB8Ypqp7RcR3Fo6jqto/wHEbY4ypBX+u9AcBeaq6SVVLgVeBy07a5hbgKVXdC6CquwIbpjHGmEDwJ+m3B7b5fM53l/nqBnQTkc9EZLGIZPmsixWRZe7yy88xXmOMMecgUJ2zooB0YBSQAiwUkT6qug/opKrbRaQz8JGIrFbVjb47i8hkYLL78ZCIrD+HWFoBu89h//oUSrFCaMUbSrFCaMUbSrFCaMV7LrF28mcjf5L+dqCDz+cUd5mvfOALVS0DNovIBpyTwFJV3Q6gqptEZAEwADgh6avqNGCaPwHXRESWqWpmIL6rroVSrBBa8YZSrBBa8YZSrBBa8dZHrP407ywF0kUkTURigGuBk6tw3sK5ykdEWuE092wSkRYi0shn+TBgDcYYYzxR45W+qpaLyB3AbCASmK6qOSLyELBMVd9x140XkTVABXCPqhaLyFDg/0SkEucEM9W36scYY0z98qtNX1VnAjNPWvaAz3sF7nZfvtt8DvQ59zDPSkCaiepJKMUKoRVvKMUKoRVvKMUKoRVvnccqTr42xhgTDmwYBmOMCSMNJumLyHQR2SUi2V7HUhMR6SAi832GrfiJ1zGdjojEisgSEfnKjfU3XsdUExGJFJEvReQ9r2OpiYhsEZHV7jAly7yOpyYi0lxEZojIOhFZKyIXeB3TqYhId5/hX1aKyAERucvruM5ERH7q/o1li8grIhJbJ8dpKM07IjICOAS8qKoZXsdzJiLSFmirqitEpCmwHLg8GB9yi4gA8ap6SESigU+Bn6jqYo9DOy0RuRvIBJqp6qVex3MmIrIFyFTVkKgjF5F/AJ+o6vNuNV+c2x8naLlDyWwHBqvq117Hcyoi0h7nb6uXqh4Vkf8AM1X174E+VoO50lfVhcAer+Pwh6oWqOoK9/1BYC3f7OUcFNRxyP0Y7b6C9kpBRFKAS4DnvY6loRGRBGAE8DcAVS0N9oTvGgNsDNaE7yMKaCwiUUAcsKMuDtJgkn6oEpFUnA5rX3gbyem5zSUrgV3AXFUN2liBx4F7gUqvA/GTAnNEZLnbMz2YpQFFwAtu89nzIhLvdVB+uBZ4xesgzsTtxPoosBUoAPar6py6OJYlfQ+JSBPgdeAuVT3gdTyno6oV7kipKcAgEQnK5jMRuRTYparLvY7lLAxX1fOAicDtbjNlsIoCzgOeUdUBwGHglEOtBwu3CWoS8JrXsZyJiLTAGcgyDWgHxIvIjXVxLEv6HnHbx18H/qWqb3gdjz/cW/n5QFZN23pkGDDJbSd/FbhIRF7yNqQz8xmmZBfwJs6otsEqH8j3udObgXMSCGYTgRWqutPrQGowFtisqkXucDZvAEPr4kCW9D3gPhz9G7BWVR/zOp4zEZEkEWnuvm+MM6/COm+jOjVVvV9VU1Q1FeeW/iNVrZOrpUAQkXj3QT5uM8l4IGirz1S1ENgmIt3dRWMI/mFVriPIm3ZcW4EhIhLn5ocxOM/6Aq7BJH0ReQVYBHQXkXwRudnrmM5gGPBdnCvRqpKyi70O6jTaAvNFZBXOOExzVTXoSyFDRBvgUxH5ClgCvK+qszyOqSZ3Av9yfx/6A7/1OJ7Tck+k43CumoOae/c0A1gBrMbJzXXSO7fBlGwaY4ypWYO50jfGGFMzS/rGGBNGLOkbY0wYsaRvjDFhxJK+McaEEUv6xhgTRizpG2NMGLGkb4wxYeT/AV3Ttuc0ziWRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "K is taken as 6\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "import matplotlib.pyplot as plt\n",
    "n_neighbors = [1,2,4,6,8]\n",
    "train_score = []\n",
    "test_score = []\n",
    "print(\"KNN\")\n",
    "for i in n_neighbors :\n",
    "    clf1 = KNeighborsClassifier(n_neighbors=i)\n",
    "    clf1.fit(X_train, y_train)\n",
    "    trainScore = clf1.score(X_train,y_train)\n",
    "    testScore = clf1.score(X_test,y_test)\n",
    "    train_score.append(trainScore)\n",
    "    test_score.append(testScore)\n",
    "    print(\"Train set score: {:.3f}\".format(trainScore))\n",
    "    print(\"Test set score: {:.3f}\".format(testScore))\n",
    "    \n",
    "plt.plot(n_neighbors,train_score,label=\"train\")\n",
    "plt.plot(n_neighbors,test_score,label=\"test\")\n",
    "plt.legend()\n",
    "plt.show()\n",
    "print(\"K is taken as 6\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Multinomial Naive Bayes\n",
      "Train set score: 0.877\n",
      "Test set score: 0.903\n"
     ]
    }
   ],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "clf2 = MultinomialNB()\n",
    "clf2.fit(X_train, y_train)\n",
    "print(\"Multinomial Naive Bayes\")\n",
    "print(\"Train set score: {:.3f}\".format(clf2.score(X_train,y_train)))\n",
    "print(\"Test set score: {:.3f}\".format(clf2.score(X_test,y_test)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Multi Class Logistic Regression\n",
      "Train set score: 0.991\n",
      "Test set score: 0.944\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "clf3 = LogisticRegression(C=10)\n",
    "clf3.fit(X_train,y_train)\n",
    "print(\"Multi Class Logistic Regression\")\n",
    "print(\"Train set score: {:.3f}\".format(clf3.score(X_train,y_train)))\n",
    "print(\"Test set score: {:.3f}\".format(clf3.score(X_test,y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}