cap5.2 - Model Evaluation and Improvement (Grid Search).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "protective-dodge",
   "metadata": {},
   "source": [
    "# Grid Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dressed-findings",
   "metadata": {},
   "source": [
    "In this notebook I'll approch Grid Search, a method for finding the values of the important parameters of a model. Grid Search is simple. It tries all possible combinations of predefined values for each parameter. Here I'll focus on Grid Search but there are other methods for hyperparameter tuning such as (1) Genetic Algorithm, (2) Random Search and, (3) Bayesian optimization."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "instant-blues",
   "metadata": {},
   "source": [
    "As I discussed earlier, SVMs are very sensitive to the (1) scale of the dataset and (2) the parameters (specially C and gamma). Therefore I'll use SVC (SVM for classification) for hyperparameter tuning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "qualified-technology",
   "metadata": {},
   "source": [
    "*C* is the regularization parameter. A small C means that individual data points have little influence on the decision boundary (low complex model).<br>\n",
    "*gamma* constrols the width of the Gaussian kernel. A small gamma means that more points will be considered close by."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "smaller-colony",
   "metadata": {},
   "source": [
    "## Simple Grid Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "hollywood-incident",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:28:04.430010Z",
     "start_time": "2021-03-21T15:28:04.344346Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score 0.97\n",
      "Best paramaters {'C': 100, 'gamma': 0.001}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "iris = load_iris()\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=0)\n",
    "\n",
    "best_score = 0\n",
    "for gamma in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "    for C in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "        svm = SVC(gamma=gamma, C=C)\n",
    "        svm.fit(X_train, y_train)\n",
    "        score = svm.score(X_test, y_test)\n",
    "        \n",
    "        if score > best_score:\n",
    "            best_score = score\n",
    "            best_parameters = {'C': C, 'gamma': gamma}\n",
    "\n",
    "print(\"Best score {:.2f}\".format(best_score))\n",
    "print(\"Best paramaters {}\".format(best_parameters))       "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "peripheral-quantity",
   "metadata": {},
   "source": [
    "Okay.. Hopefully you got the idea of Grid Search, but 0.97 is an overly optimistic result because we used the test set to find the best parameters. We shouldn't use the test set for hyperparameter tuning, so we need a portion of the data to validate our parameters before we test it. Such as depicted below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "horizontal-kitchen",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:30:49.629632Z",
     "start_time": "2021-03-21T15:30:49.061808Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1MAAABqCAYAAABZJQtkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAgVklEQVR4nO2debgdVZmv3x85gYCMCbMICTgBTjTggKKxUXEAcYqCgkHbqRu0Fa6ICBKhhQu2F3C6VxshNDIZQAYBUcB4CQgIiMooBBJACYQhAyTBnOTrP9baSaVOnXP2dM6pkN/7PPvZOVXvXrWyd9Va61tDlSICY4wxxhhjjDGtsdZIZ8AYY4wxxhhjVkccTBljjDHGGGNMGziYMsYYY4wxxpg2cDBljDHGGGOMMW3gYMoYY4wxxhhj2sDBlDHGGGOMMca0gYMpY2qOpCmSQtLEDtOZmNOZ0pWMGWNe8EiaJWlWadvBuSw5uIV0pubPjO9yFsvH6ZNfY4wZShxMGdMCksbnBsHUkc6LGRj/Vsa88JA0XZIfkIk7yFplJOuEbnWKdpM65ml1pWekM2CMGZQfAOcDD3eYzi3AjsCTHefIGLMm8wvgJuCxkc5IBXuNdAaMMWsWDqaMqTkR8SRdCIAiYhFwb+c5MsasyUTEfGD+SOejioiYOdJ5MMasWXianzFNkqdSPJT/nJyHx6O4dqA47ULS6yVdIenp4loBSW+X9BNJd0taIGmxpDslHStpTNVxq4bi87bpkjbN6T0m6XlJd0n6VEU6lVNCGtNmJPVIOkrS/TmdRySdJGntfr6PT0i6Pef/CUlnS9q61Wk4kl4j6by81uF5SXNzuqdKGl1yeyT9m6Sb8ne3SNIfJR0qaa2CN4VBfitjVmckvSmfzxcP4NyTr6mx+e+187VypaTZed/Tkq6R9J4Wjt3vmilJ75B0vaTnctqXSHrlIGldJOnBXJYskHSDpANL3vhcrrwt/128pqcXvMo1U5LWkXSkpD/ncmNBzudHK9wV08Hyv8+X9KSkJZJulbRPs99VTm9PSZdLejR/53NyGXZshbuepK9LuiN/h89K+r2kA0reVOC3+c9jS9/HxFbytybQSp0gae98jTyZf6+Zkr4jaeOKdAetv/L52Pitf1s8dhP5lqTJkm7MaS9RqpuvlvSxCn8bST/I19Pzkp6SdJmk3Ute23kyffHIlDHNMx3YGPh34E/AJYV9d5TcNwFfB2YAZwCbAv/I+74GvBK4EbgCGAO8GZgCTJT0johY1mSeNgZuyGlfmNP6CHCGpOURcVaT6QCcC+wJXAUsAN4LHAFsDqwSnEn6KnAy8AxwFqmX+p05L033WEt6DXAzEMBlpMpuQ+ClwL8BRwNLszsauBzYG7gv53cJ8Hbg+8AbgINy0tNp/rcyZrUjIn4v6T5gH0njIuKp4n5JryeVMxdFxNN581jgNFLZ8xtgLrAVsC9wpaTPRsTp7eZJ0keAC0jl0QWkaYBvAX4P/Lmfj/1f4G7g/2d/HKnsOVvSKyLimOzNA74FHAxsl//dYNYg+VobuJoUiN0L/BBYj1RWXiDpdRFxVMVHtyNNj34QOJv0/X0MuDSX07+t+Ez52O8mlfMLSGXc33I6O5LKuG8V3I2B64BdgNtJdcdapDLvXEk7R8TRWb8kv08Gfkcq8xrMGixfayDTaaJOkPRN0m/yNPBL4AngNcD/At4r6U0RsSC7zdZfpwIfIJ1/Z9Ha7/NtUlviIeDnpPp1K2B3YBLpOmvk/Z+AX5POr6uBi0ltjw8AMyR9MCKuzHoneTJlIsIvv/xq8gWMJxWcU/vZPzHvD+Dz/TjbA6rYfnz+3MdK26fk7RNL2xvHOR0YVdi+E9AL3N1P3qaUtk/P228Dxha2vwh4AFgGbFnK/1JSQ+wlhe0Czmvkq8nv87vZ369i3ybAWhXfw/dL/99RwE/L6Qz2W/nl1+r+IjWyAji0Yt8P8759C9vWAbapcDcC7iQ1INct7ZsFzCptOzinfXBh2/rAU7ls2K3kn1Ior8aX9u1QkZ+1gWtzWi8u7Zs+UPnST34b39OVQE9h++bZD2CPwvbxhfweW0pr70ZaTf5GF2X/tRX7Ni39PTW7R5S2jwF+BSwHXlfYPpGKMt2vfn+LAesEUsdckDobNi7ta5zzpxS2tVN/TWwxz08BjwLrDXT+kAZHHiB1ML6t5G1NCuIfA9bpNE9+9X15mp8xQ8MdEfHjqh0R8WDkkqzEqfl97xaOswg4LAojWRFxN2mEaEdJG7SQ1tdiZQ82EfEccA6pZ3S3gvdxUsH9/Yh4pOAHcCQp+GqVxeUNEfFMRCwHUJrCdygwB/hK6f+7DDicVCl8oo1jG7O6cjapgT25uDGPxOxP6lW/qrE9Ip6PiEfLiURaA3UGqQG4e3l/k+xH6hE/NyJuLe2bQj8j1lGxxiki/kEKBnvozg0lPk0qHw6LiN7CcZ4gdWIBfKbic7OB/yjl7WrSzYBe32Ieqsq4FWthJY0DDgRujYiTS94S0owGkcpfMzR8Kb9/NiLmFXdExFTSCFZVHTNg/dUFllJRrxbPH+B9wA6kevl3Je/vpJkkW+IbtAwJnuZnzNBwS387JL2INNXgg8DLgQ1IlWSDF7dwnPsjTzko0QhyNgYWNplWuQFUTGeTwrZd8vuMshwRsyU9QuoBbIYLSN/FJZIuBK4BbqhoYL2cNP3nfuBoSVSwmDR1xpg1goh4VNK1wDsl7ZQ7UiBN2xtL6kXvLX5G0s7AV4G3kqYLlddptlL+FPmn/P678o6ImC/pDvJ6p1J+tiUFCnsB2wLrdik/jfQ3IE27+ltEVN2A57r8vkvFvjuiesr1I6Sp3M1wDvAh4GZJF5DWOd1QEdTuThpl7+9W5431oy7jho43kQKXSZImVexfG9isMK222fqrE84BvgjcJWka6fr6fe4AKecdYLt+zp+X5fcdSSO0pos4mDJmaJhTtTGv+7mO1Kt5J6kwnkteF0RaELpOC8eZ18/2RgNqVLMJlXviBkhno/z+eD9JPU6TwVRE3CJpT+AbpPULBwHktSDfiojzsjouv7+MlYtmq1i/meMa8wJiKmm94mRSUAIrR6pWWTMp6Y2k8qeHNI3uMtJanuXA60ijS62UP0UGKxf6lImStid1PG0CXE9a7zGf1As/Pv8/2s1POV/93ca9sX3jin3z+vlML03ewCsiLs43rDicNEL2eQBJtwFfj4jfZLVRxu3OwKODLuOGjnGka2OgOgbylNYW6q9O+Aowk3TuHJlfvZKuBA6PiAcKeYe0jmqwvJsu42DKmKGhvzvi7EcKpM6KiIOLOyRtxeCFeB1ojIRtAdxVsX+LVhKLiN+TFtGvA+wKvJvUE3eupLkRcQ0rpwj9IiI+1F62jXlB8gvSNXmgpKNII1LvAf4UEX8quUeTRn7eHhHTizskfZ1UPrVL4xrt7/rfsmLbYaRG4KfyNKpifg6gNH2xw3xVHR/S6FzR6zoRcQVwRZ6V8AZgH+BfgV9K2iWPKDaOf0pEHDZUeTEDMp+0zmlssx9osv5qmzwyehpwmqTNSTd02Z8UNO2cb0ryPCvPn/0i4rJOjmlax2umjGmNxpSPpkd8Srw0v19Usa/PFJia8sf8/pbyDknbAS9pJ9G8nuPGiPgmK+euNxp395J6id+o0u3SB6DT38qY2hMRi0l3+doaeAdpTUcPpVGpzEuBp8uBVKbT8uf2/tKRtBFp5KsqP9Baebgsp9nUdR0RC0k9+y+W9LIK5e35/faKfV0lIp6LiOtysHQCadpY45b0t5BGCPdsIUmXca0x2Pd1E7BJngrbEoPUX80cu5ljPBERF0fER0kjzDsAr8q7b8rvPn9GAAdTxrTGM6RRp23b/Pys/D6xuDFPdzmp7VwNL+eSprl8UdKKwElpIdOJtFAwKz1/ZaOKXY3e7UUAed3H90m9yN+TVF5XgaStJO1U2NTpb2XM6sLU/P7J/OolrbUoMwsYm2/pvAJJ/0JrN76p4lLSNfdxSbuV9k1h5XS7cn6gb3m4N9U3hIB0dzNo7bo+g7Qu9TvFIEzSpsAxBafrSNqrqryibxn3BOk3203SMZL6zByStIOkCYVN7XwXazKD1Qmn5Pf/krR1eaekF+Wpso2/m6q/Mi3/VkrPRttLpUXCuUOxMXrWOMalpE6DQyS9t5/03iRpvU7yZKrxND9jWiAinpV0M7CnpHOAv5J6dy6LiP6eo1LkctLtSw+T9GrSKM+2pGkfV7AaFGoRMTM/i+ME4E95UXXjOVNjSc/weM0ASRQ5HHiX0kM3HwSeBXYm9dY+A/yk4B4PvBb4ArCvpOtIt3vdnLSW6s2kuet353x2+lsZs1oQETdIeoA09Wc0cHlunJc5lRQ0zZDUeGbNbqRR5gtJ6z7azcOzkj5HWgd6fS4XGs+ZehXpOVJvLX3sR6Rn2E2TdBHpen4VaarUz0nPdCpzbf5/XpzXjSwGZkfE2QNk7z9JZcp+pDLrStJzpiaRyo+TI6LPDXW6xHeB8bmMm0V6BteuwD+T7hZ4fsE9lFSWHQccJGkGaQ3a1qQbB+wOHMDKh8/eR/rO9pf0D9JdBgM4OyJmD9H/Z7VlsDohIq6VdCSpU/D+fJ48RFpntB1ptHQG6fyE1uqv35JGHk+U9Kq8n4hY5W6RJdYl3dRiVs73bNINY95JOh8ui4h7cjpLJX2I9HypKyTdSLr74CLSbJHdSY812YqVAVg7eTJVdHpvdb/8WtNepKkpl5N6dZZTeN4KTTz3g1SwnUOqBBeT1h0dQercCGB6yZ9C/8+Zmt7PMaZSeqZLf3ljgOe2UPE8mcK+g0jB4BLSTTR+Rqr07wTmNfldvgs4kxQAzQeeIzUQvgdsV+ErH/da0jNx/pG/xxnAURSeezXYb+WXXy+kF2k9VOPZSB8ewNuHNCVoIWnq7K9JQU7ltU6Tz5kq7Htnvh4XkRpnl5IeHtynTMr+HqQpS8/kPM0gPUy0v/JqFKkj50HSjXtWKQer8pu3j8llxJ2kcrdxrAMq3PEM/DyifsvMCvejpOfv3U9qbC/Iefg2sFmFvzYpqLoxl4nPk4Kka4EvA+NK/u553/xCGTdxpM/Hur6aqRNIHQA/B/6e65i5pMDk/1B4hhqt118H5nQW08TzGEkdI0eQHm/wMCvr2ptInYprV3xmc+B/53NsUT7n7id1lhxI4Tlr7eTJr+qX8pdpjDEdI2lDUk/qHRHR7K2DjTHGGGNWS7xmyhjTMpI2K98IIs/x/y6pB/gXI5IxY4wxxphhxCNTxpiWkfQF0rz+a0gPsBxLmir0ctKUgT0i3WXMGGOMMeYFi29AYYxph5tJ6w3eysqHBT5EWgdwkgMpY4wxxqwJeGTKGGOMMcYYY9rAa6aMMcYYY4wxpg0cTBljjDHGGGNMG7S0Zmrdddeds2TJki0GN40xxhgzkowZM+bxxYsXb1ncNnr06Md7e3s3H6k8maGlp6fniaVLl65op40ePXpOb2+v223GDEJPT8/jS5cu3XJwsy8trZmSFGV/+vTpTJo0iWnTpjFx4sRB07Bv3759+/btD70viYhQaVtMnjyZCRMmDJr+Qw89xLRp05g0aZL91cQ/66yzVvnNJcWUKVMG/awxazpTpkzpU142S0fT/Ea6orBv3759+/btD+4XqUvD3373fWPM8NN2MFW3isK+ffv27du33zl1DRTsD+4bY4aftoKpulUU9u3bt2/fvv3OqXOgYL9z3xjTfVoOpupWUdi3b9++ffv2O6duDX/73fWNMUNDy8FUnSoK+/bt27dv33613wp1a/jb765vjBk6Wg6m6lRR2Ldv3759+/ar/WapW8Pffnd9Y8zQ0nIwVaeKwr59+/bt27df7TdD3Rr+9rvrG2OGno5ujV5FnSsW+/bt27dv336ibg1/+931jTHDQ1eDqbpVFPbt27dv3779vtSt4W+/u74xZvjoWjBVt4rCvn379u3bt9+XujX87XfXN8YML10JpupWUdi3b9++ffv2q6lTw99+d31jzPDTcTBVt4rCvn379u3bt98/dWn42+++b4wZfjoKpupWUdi3b9++ffv2B6YuDX/73feNMcNP28FU3SoK+/bt27dv337n1DVQsD+4b4wZftoKpupWUdi3b9++ffv2O6fOgYL9zn1jTPdpOZiqW0Vh3759+/bt2++cujX87XfXN8YMDS0HU3WqKOzbt2/fvn371X4r1K3hb7+7vjFm6Gg5mKpTRWHfvn379u3br/abpW4Nf/vd9Y0xQ0vLwVSdKgr79u3bt2/ffrXfDHVr+Nvvrm+MGXq68tDeInWuWOzbt2/fvn37ibo1/O131zfGDA9dDabqVlHYt2/fvn379vtSt4a//e76xpjho2vBVN0qCvv27du3b99+X+rW8LffXd8YM7x0JZiqW0Vh3759+/bt26+mTg1/+931jTHDT8fBVN0qCvv27du3b99+/9Sl4W+/+74xZvjpKJiqW0Vh3759+/bt2x+YujT87XffN8YMP20HU3WrKOzbt2/fvn37nVPXQMH+4L4xZvhpK5iqW0Vh3759+/bt2++cOgcK9jv3jTHdp+Vgqm4VhX379u3bt2+/c+rW8LffXd8YMzS0HEzVqaKwb9++ffv27Vf7rVC3hr/97vrGmKGj5WCqThWFffv27du3b7/ab5a6Nfztd9c3xgwtLQdTdaoo7Nu3b9++ffvVfjPUreFvv7u+MWbo6cpDe4vUuWKxb9++ffv27Sfq1vC3313fGDM8dDWYqltFYd++ffv27dvvS90a/va76xtjho+uBVN1qyjs27dv3759+32pW8Pffnd9Y8zw0pVgqm4VhX379u3bt2+/mjo1/O131zfGDD8dB1N1qyjs27dv3759+/1Tl4a//e77xpjhp6Ngqm4VhX379u3bt29/YOrS8Lfffd8YM/y0HUzVraKwb9++ffv27XdOXQMF+4P7xpjhp61gqm4VhX379u3bt2+/c+ocKNjv3DfGdJ+Wg6m6VRT27du3b9++/c6pW8Pffnd9Y8zQ0HIwVaeKwr59+/bt27df7bdC3Rr+9rvrG2OGjpaDqTpVFPbt27dv3779ar9Z6tbwt99d3xgztLQcTNWporBv3759+/btV/vNULeGv/3u+saYoacrD+0tUueKxb59+/bt27efqFvD3353fWPM8NDVYKpuFYV9+/bt27dvvy91a/jb765vjBk+uhZM1a2isG/fvn379u33pW4Nf/vd9Y0xw0tXgqm6VRT27du3b9++/Wrq1PC3313fGDP8KCKaltddd905S5Ys2WII82OMMcaYLjBmzJjHFy9evGVxW09Pz1PLli0bO1J5MkPLqFGjnu7t7R3X+Hv06NFzent73W4zZhB6enoeX7p06ZaDm31pKZgyxhhjjDHGGJPo+t38jDHGGGOMMWZNwMGUMcYYY4wxxrSBgyljjDHGGGOMaQMHU8YYY4wxxhjTBg6mjDHGGGOMMaYNHEwZY4wxxhhjTBs4mDLmBYSkXkkTm3QnSuodxDlS0uOSnpW0u6SrJB3RRr6OknR5q58zxpiRRNJ0SUcPYfouG19gSApJbxniY9wl6WNDeQzTPA6mjBlGcsUckj5a2v6GvH3WCGWtD5K2AU4A3h4R60fEHyLiPRFxcsHpU2lUNT4i4oSI2Hd4cm7MmkW+5p7PnR7zJf1R0odHOl+tIGmKpGtGOh9DicvG+lC6ZoqvV4903opIGp/r2W2K2yNi54i4YKTyZVbFwZQxw889wGdL2z6bt9eJ8cDyiLh7pDNijBmU4yNifWAccB5wgaSXt5qIpNFdz9kwsTrn3YwIx+eOwuLrLyOdKbP64WDKmOHnYmAXSdsDSNoA+DBwZlGStJ6k0yQ9IulJSZdI2rawfwNJZ0l6WtJsSZPLB5L0AUm3SZon6R5Jn2gmg3n6wG+AUbm3bmbevqJnVdKfsv7r7Jwu6QfAnsAxedt92V2l11nSrDy95drs3Slpj8L+0ZJOkfSEpDmSjpD0gKSDm8m/MWsqEdEL/AgYBbxa0l6Sbpb0jKS5ks6XtHnDz9f0qbl8WQAcLmkbSb/K/nxJ10vatfCZKfnaPSk7T0k6TNJ2kq6TtDCXOzsWPtOTr/m/5vLohkaaubw5CphYGCFolI97SpqRy7mZkg6XpLxvotLU5oMkPQg8XfWdSNo/l38LlaYtTy3sGyfpp7mcnSvp55K26O/7lbStpAslPZZfP8lleGP/Zjm9hyUtyN/DK1ooG8dJ+u+c9pxcxo8t7B+w7DSdIWmfXO+MLmxbP3/Xb81/nyDpwUbdKOnLA6R3sKQHStumSjq98PeZ+fxbKOluSR8v6I169r58vGPyZ2ZJOrCQxtvydT5f0r2SPl/Y17hOPpbzOz+f5xtguoKDKWOGnyXAOcC/5L8PAH4HPFbyTgHemF/bAU8Cl0salfefCrwM2Al4DbAfqQEFgKR3Aj8FvgyMBSYDP2hUCAORpw+8B1iWe+t2qHBem//5rux8JiIOBa5nZY/fKwY4zKeBLwEbkQK3swr7vp6P/0ZgArBN/g6MMQMgaW3gEGApqSH2PHAosBnwamBr4LTSxz4NfI90LX6P1Db4Eema2xK4HbhYq478vBW4P+8/EPgOqbw5hFTe3FM6znGkMurdpNGzM4CrJW2Sy5sTgOmFEYIHJe0MXJnT3gx4X/6/HFRIdxSprNgF6BMESVoPOBs4JCI2ALbP+SQHZZcAAbwq/38XAuf2892OAa4D7s7p7EQqm07L+9cCLgU2BnbP758CFrZQNp4DbJLT3hHYNOe/yEBlp+mMq4Be0rnWYBIwh/T7Qfr93wJsQJpVcqKkvTs45gzgdaTz5ThgqqSd8r5GPfuKfN4cX/6wpAnAr4D/R7q2Ds55mlTQRgHvyum9nHS9fKmDPJsCDqaMGRn+C/iUpB7gc/nvFeRK+ZPA0RHxt4h4jhQU7Qi8Pu//BHBMRMyJiPnA10rH+HfgtIi4PiKWR8QtwM9yunXgxxFxV0QsA04HXippo7zvk8DJEfFgRCwm/d+Wj1RGjVkN+IakecCjpKDlwxHxQETMyOsdeyNiDnAysFfpsxdGxHWRWBQRD0fEZfnfi4GjgW1JnTcN/hoRp0fEsoi4CngKuDoi7omIpaSAZHdYEbR8EfhqvqaXRcRPSR1IxUZrmX8FpkXEpfkz9wI/oG8ZdmREzI+IRf2ksxR4paSxEfFcRDQaxbvm1yGFzx8B/LNKa1Qy+wCKiG9GxOKIeAY4BvhE7uTaLf+fPx0Rj+dy988R8fcB/o8rkLQ1sDdwWEQ8k9M/DHivpK0K6kBlp2meb+RR0hWv/J2eTQqCG3wKODMiAiAifhYRf8/Xy3XAFfS9ppomIn4aEU/lc/x84M/AxBaSOAC4PSLOzNf5TcCPgc+UvCMj4tmIeJzUibBbu3k2q+JgypgRICLuBGaTKuItSL1KRTYDxgAPFj7zLPAE8JK8fx1gVuEzD5XSmAB8rVhRkHqstu7W/6NDiiNxz+X3xrSDF5O+HwByg27uMOXLmNWRb0fExhGxeUTsERGXA0jaVdLVecrYAtJ6qs1Kn51V/EPSpnmq2cP5M4/kXcXPlUfSF5W2LWLl9bwpsD5pZL1YHm1PGtnpjwnAAaXPHAsUA4vlhfz1IQdI7yWNiM3M0+4a06gmkMrRxwvpzyTNHti2IrkJwLal/FxLGtnakrTO9IncudUOL8nvxbJ8ZmkfDFx2muZpXDMrXnn7mcB7JG0uaQdgDwqjf5K+JOkvSlNn5wH70veaagpJa0k6TtJ9efrdPNLoUSvpvYRCWyEzk1XPmWURUaxDn8PnTNfoGekMGLMG8xPSdJPjImJZ6rxdwVzS9JwJ5MpU0vrA5qSGw1zgH6TKu1HZTiilPxuYGhHfGaL8Q2pElOnGCNLfKEzrk7QubVZWxqzhnA9cCEyKiAWS9gHKt+IuX7MnkgKWN0TEY3ltxQJAtMeTpMbbOyLiD/04VeXGbOCMiDhkgLSjMWIwgDAdmJ5Hj94PXCTp5pz+c8DYiGim3JpNGpHbuWqn0t1YN5e0YUQsqFAGO0YjKBwPNNbZbF/aZ4aYiLhX0m2k6aubANdExKMAkt4MnEQaibo5190X0v+18SzwotK2rYGH878PII0gvQu4OyKWS7q1kF4z5+UjpA6DItvjc2bY8MiUMSPHeaQCtLx+gVyx/zdwvKSt87z/7wL3Arfk/ecC35K0haQNSQ2gIqcCX1ZawD1K0tq5l7qbQ/tzWHXqT2PbSztM92zgq5Im5HUKJ+Lyyph22BCYDyxUuoHNkU1+ZhHwTO7EOamTDORg5zTgPyW9DFYs6t87T22DVG5sm9d8NfgRsL+kfZVuStMjaSdJb2v22Ll8/LCkjfIUrnl51zLgVuAO4DRJ47K/maT9+0nul8BopRtAbKDEiyV9MO+/FbgNOD2Paqwl6dWFKXoDlo15OuCvge9K2ljSJqRy/6qIKI8EmqHlTNLatE+S1vc12JB07swFQtL7SGv2+uOPpAB7n3w+fJC03rCYXm9Oby1Jn2blOiny9uX0rWeLnAfsKumT+Rp5PfB58tpAM/S4cWLMCBERSyLimjwvvoqvkCrnP5B6sbYC3p8bBJDWRD1ECrD+QuptbuwjIn5NWo/1HVLP8GOkm1qs38X/xjeA4/J0hx/nbacAu+VpMHe1me6JpIXVt5CmID0G/J00WmeMaZ7PkXq+F5LuJDqtic8cSxoFf4q0fuNGCmVLmxxLujnDpXnq4P3AF1jZDplG6kmfk8uOCXk69D6k9aKPkaY5T6W1Ueq1SDfFmCVpIfBDYHJEzMqdUh/Izm15/830s14lTxnci3RziHtJQeq1pJsHNDrB3g8sJgVp80iN8sZ0qmbKxgNJv9W9+TWP+qxzfaHRuLNi8bVP3nc+aXRnfdJ52+BqUmffLaR69SPAL/o7QETMJNXVPyHdbfLdwEUF5SzSOfcAaUbGTqy80UVjivsxwHn5vPlGxTEeIo1MHUq6Zs8GvhkRP2/2izCdoUFGx40xZsTJvePPAG+LiBtHOj/GGGOMMeCRKWNMDZG0iaR356k9jds1zyaN0hljjDHG1AIHU8aYOjIK+A/StIiHSHf82jffctkYY4wxphZ4mp8xxhhjjDHGtIFHpowxxhhjjDGmDRxMGWOMMcYYY0wbOJgyxhhjjDHGmDZwMGWMMcYYY4wxbeBgyhhjjDHGGGPawMGUMcYYY4wxxrTB/wDA9r6w+sbxMgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x72 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import mglearn\n",
    "mglearn.plots.plot_threefold_split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "first-munich",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:39:13.234995Z",
     "start_time": "2021-03-21T15:39:13.164267Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score on validation set: 0.96\n",
      "Best parameters:  {'C': 10, 'gamma': 0.001}\n",
      "Test set score with best parameters: 0.92\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "iris = load_iris()\n",
    "\n",
    "X_train_val, X_test, y_train_val, y_test = train_test_split(iris.data, iris.target, random_state=0)\n",
    "X_train, X_validation, y_train, y_validation = train_test_split(X_train_val, y_train_val, random_state=1)\n",
    "\n",
    "\n",
    "best_score = 0\n",
    "for gamma in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "    for C in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "        svm = SVC(gamma=gamma, C=C)\n",
    "        svm.fit(X_train, y_train)\n",
    "        score = svm.score(X_validation, y_validation)\n",
    "        \n",
    "        if score > best_score:\n",
    "            best_score = score\n",
    "            best_parameters = {'C': C, 'gamma': gamma}\n",
    "\n",
    "            \n",
    "svm = SVC(**best_parameters)\n",
    "svm.fit(X_train_val, y_train_val)\n",
    "\n",
    "test_score = svm.score(X_test, y_test)\n",
    "print(\"Best score on validation set: {:.2f}\".format(best_score))\n",
    "print(\"Best parameters: \", best_parameters)\n",
    "print(\"Test set score with best parameters: {:.2f}\".format(test_score))    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sacred-valuation",
   "metadata": {},
   "source": [
    "The distinction between the training set, validation set and test set is fundamentally important to applying machine learning methods in practice."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bound-response",
   "metadata": {},
   "source": [
    "It is a good practice to do all exploratory analysis and model selection using the combination of a training and a validation set, and reserve the test set for a final evaluation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "documentary-people",
   "metadata": {},
   "source": [
    "## Grid-Search with Cross-Validation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "subtle-significance",
   "metadata": {},
   "source": [
    "Simple Grid-Search inherits the problems of train-test split. So, our results are very dependent of the selected data. We can solve this addind cross-validation to Grid-search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "polish-potter",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:46:49.163965Z",
     "start_time": "2021-03-21T15:46:48.803485Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score on validation set: 0.97\n",
      "Best parameters:  {'C': 10, 'gamma': 0.1}\n",
      "Test set score with best parameters: 0.97\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "iris = load_iris()\n",
    "\n",
    "X_train_val, X_test, y_train_val, y_test = train_test_split(iris.data, iris.target, random_state=0)\n",
    "\n",
    "best_score = 0\n",
    "for gamma in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "    for C in [0.001, 0.01, 0.1, 1, 10, 100, 1000]:\n",
    "        svm = SVC(gamma=gamma, C=C)\n",
    "        score = np.mean(cross_val_score(svm, X_train_val, y_train_val, cv=5))\n",
    "        \n",
    "        if score > best_score:\n",
    "            best_score = score\n",
    "            best_parameters = {'C': C, 'gamma': gamma}\n",
    "\n",
    "            \n",
    "svm = SVC(**best_parameters)\n",
    "svm.fit(X_train_val, y_train_val)\n",
    "test_score = svm.score(X_test, y_test)\n",
    "\n",
    "print(\"Best score on validation set: {:.2f}\".format(best_score))\n",
    "print(\"Best parameters: \", best_parameters)\n",
    "print(\"Test set score with best parameters: {:.2f}\".format(test_score))    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "national-shape",
   "metadata": {},
   "source": [
    "The main downside here is the number of models we need to evaluate our model. In this case, we need (7*7) * 5 = 245 models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "pediatric-elite",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:56:23.021048Z",
     "start_time": "2021-03-21T15:56:22.666689Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set score: 0.97\n",
      "Best parameters: {'C': 10, 'gamma': 0.1}\n",
      "Best cross-validation score: 0.97\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000], 'gamma': [0.001, 0.01, 0.1, 1, 10, 100, 1000]}\n",
    "\n",
    "grid_search = GridSearchCV(SVC(), param_grid, cv=5)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=0)\n",
    "\n",
    "grid_search.fit(X_train, y_train)\n",
    "\n",
    "print(\"Test set score: {:.2f}\".format(grid_search.score(X_test, y_test)))\n",
    "print(\"Best parameters: {}\".format(grid_search.best_params_))\n",
    "print(\"Best cross-validation score: {:.2f}\".format(grid_search.best_score_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "vertical-reproduction",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:57:20.486075Z",
     "start_time": "2021-03-21T15:57:20.472354Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best estimator:\n",
      "SVC(C=10, gamma=0.1)\n"
     ]
    }
   ],
   "source": [
    "print(\"Best estimator:\\n{}\".format(grid_search.best_estimator_))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "referenced-newark",
   "metadata": {},
   "source": [
    "Let's see all Grid-Search results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "falling-robin",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T15:58:43.615727Z",
     "start_time": "2021-03-21T15:58:43.571340Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_C</th>\n",
       "      <th>param_gamma</th>\n",
       "      <th>params</th>\n",
       "      <th>split0_test_score</th>\n",
       "      <th>split1_test_score</th>\n",
       "      <th>split2_test_score</th>\n",
       "      <th>split3_test_score</th>\n",
       "      <th>split4_test_score</th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>rank_test_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000788</td>\n",
       "      <td>0.000490</td>\n",
       "      <td>0.000198</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>{'C': 0.001, 'gamma': 0.001}</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.001098</td>\n",
       "      <td>0.000202</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.01</td>\n",
       "      <td>{'C': 0.001, 'gamma': 0.01}</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000798</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000488</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.1</td>\n",
       "      <td>{'C': 0.001, 'gamma': 0.1}</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000598</td>\n",
       "      <td>0.000488</td>\n",
       "      <td>0.000392</td>\n",
       "      <td>0.000480</td>\n",
       "      <td>0.001</td>\n",
       "      <td>1</td>\n",
       "      <td>{'C': 0.001, 'gamma': 1}</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000595</td>\n",
       "      <td>0.000486</td>\n",
       "      <td>0.000402</td>\n",
       "      <td>0.000493</td>\n",
       "      <td>0.001</td>\n",
       "      <td>10</td>\n",
       "      <td>{'C': 0.001, 'gamma': 10}</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   mean_fit_time  std_fit_time  mean_score_time  std_score_time param_C  \\\n",
       "0       0.000788      0.000490         0.000198        0.000397   0.001   \n",
       "1       0.001098      0.000202         0.000199        0.000397   0.001   \n",
       "2       0.000798      0.000399         0.000399        0.000488   0.001   \n",
       "3       0.000598      0.000488         0.000392        0.000480   0.001   \n",
       "4       0.000595      0.000486         0.000402        0.000493   0.001   \n",
       "\n",
       "  param_gamma                        params  split0_test_score  \\\n",
       "0       0.001  {'C': 0.001, 'gamma': 0.001}           0.347826   \n",
       "1        0.01   {'C': 0.001, 'gamma': 0.01}           0.347826   \n",
       "2         0.1    {'C': 0.001, 'gamma': 0.1}           0.347826   \n",
       "3           1      {'C': 0.001, 'gamma': 1}           0.347826   \n",
       "4          10     {'C': 0.001, 'gamma': 10}           0.347826   \n",
       "\n",
       "   split1_test_score  split2_test_score  split3_test_score  split4_test_score  \\\n",
       "0           0.347826           0.363636           0.363636           0.409091   \n",
       "1           0.347826           0.363636           0.363636           0.409091   \n",
       "2           0.347826           0.363636           0.363636           0.409091   \n",
       "3           0.347826           0.363636           0.363636           0.409091   \n",
       "4           0.347826           0.363636           0.363636           0.409091   \n",
       "\n",
       "   mean_test_score  std_test_score  rank_test_score  \n",
       "0         0.366403        0.022485               28  \n",
       "1         0.366403        0.022485               28  \n",
       "2         0.366403        0.022485               28  \n",
       "3         0.366403        0.022485               28  \n",
       "4         0.366403        0.022485               28  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "results = pd.DataFrame(grid_search.cv_results_)\n",
    "\n",
    "display(results.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ordered-production",
   "metadata": {},
   "source": [
    "The dataframe contains too much information. Let's visualize it with a Heatmap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "operational-treasurer",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T16:01:04.216516Z",
     "start_time": "2021-03-21T16:01:03.914182Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARkAAAEGCAYAAABRkOFZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABFcklEQVR4nO2deXgURfrHPzVXAiEhhBAySQgBEgLhRuR2RSUcAoIHElBYXFxkFVQUdVVWvFEWBBUvXEBQgaj7Q1jkEDFeCAqrKMoZDAmT+5wQQjIzPfX7Y4aECQEmmCG41Od58sxM1Vvz7eru+XZVd7pfIaVEoVAofIWuoRdAoVD8b6NMRqFQ+BRlMgqFwqcok1EoFD5FmYxCofAphoZegItBaIheRre6LLrqgZOGu3LYkNoVzobb1pa80AbTNuSdaDDt4xQXSClb1FZ3WfzyolsZ2L45oqEX46JTLu0NqK01mPYBW3CDac9aOLXBtMMWf9tg2p/Jj9LPVqemSwqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqcok1EoFD7lsrit4Gx8mlLOQ/8oQnPC5PFNmDUj2KO+uERj2gOFpKXb8fMTvPlSKJ06mAAosWrcPauQfQdsCCF486Xm9Onl/4fQ3pZSwWNPWHE6JbePD+C+6YEe9SUlTu59sJij6Q78/ASvLGhGxw5GAHr0yaFJE4FeJ9AbYNumMK91Ab5IqeTJOaVoGiSNb8Q905ucof3Qg1bS0zX8/ATzFwQR79bu3zePgAAdej3oDfDJxrrdJ7TryzLeeDoPp1My7NZgkv7W3KP+uFVjwSPZZKfbMfkJHnjRTJt4PwAWPJzNzpQygpvreXtz2zrpAvTv2JpHbhqETqdj7Y5fWPbZLo/6XrFRLPrrDWQWWgH4/OdU3tr8HQC3D+rBTf26IKXkcHYBT7z/KTaH97dt9BranbsX3YFOr2PT0m0kv/ixR32/G3ox+ekkpFOiOTRen/kOv24/QFT7CGavmVkVF942jBVzkln78sY69f2imYwQYhkwEsiTUnZ2l4UAyUAMcBS4VUpZ7K57FJgCaMC9Usot7vIrgHeARsBG4D55Ac8Q1TTJzMeK2LCmJZFmA1ddn8WIoY3p2N5UFfPPV6x07WQieVkYBw/bmPl4ERs/CAfgoSeKSBzUiFVvh2GzScpPOv8w2o88XsJHq0OJMOtJvD6PYUP8iW9vrIpZ+OpxOncysnJpcw6n2nn4MStrP6j+QX/8YSjNQ/Rea56uPXt2Ke+vaobZrGfUiEISh/jTvn31bvjaq2UkdDLy9tJmpKY6mP14KWuSQ6rqkz8MISSk7gNwTZMsnpPLCytbERpuZMaYo/Qb3ITWcX5VMatfL6RdR3+efDOKjCOVLH4il3nvRwOQeEtTbpjUjHmzsuqsrROCx8Zey12v/R+5JcdZNWsCX/xyhN9yijzifjySyYwl6zzKwpoGMOHqHtz4/Aoq7Rrz7hjBsJ7xrP9+n3faOh0zFk/hkSHPUGApYvH3c9mxfjcZ+y3Vutt+Ycf6WQC06RLN7OQHmJJwP5ZDWUzr+VDV96y2vMX2td/Xvf91bnHhvAMMq1H2d2CblDIO2Ob+jBAiAUgCOrnbvC6EOLVXvwFMBeLcfzW/0yt2/1hJuxgDbVobMZkEt4wOYMOWco+Y/YftXDPQNUKIjzORfsxBbr5G6XEn3+ysZPIE11HYZBIEN/X+R9eQ2j/8aKNNjIGY1gZMJsGNoxuzaUuFR8zBQ3b+NND144uLNXLM4iAv//ff8Lhnj52YGD2t3dqjRvvz6aee2ocPawwY6DLb2FgDFotGfj1oH/ypgojWJszRJowmwdUjg/h2a5lHTMbhSnr0bwxAdDs/cjPtFOc7AOjauzGBwRf2c+ncOpxj+SVkFlpxaE42/3CQQV3aed1er9PhZzSg1wkaGQ3kl5adv5Gb+N6xZKXmkJOWh8Pu4Ivk7fQf3csjpuJE9TbwD/CHWo7ZPa7rTPaRHPIyCrzWPsVFMxkp5VdAUY3i0cAK9/sVwJjTytdIKSullGlAKtBbCGEGgqSUO9yjl5WntakTWTkakRHVR9BIs4GsbM+duUuCiXWbXD/+XT9WkmFxkJntIC3dTmhzHXfNLKBvYhZ/e7CAE+XejyYaUjs7x0lERLUpRZj1ZOd4andOMLJh40nAZUrHLFrV8gkBt4wv5Npheax4r26PFsjJdhJhrtY2h+vJzfZc9o4JBjZvcu30e360kWnRyHbHCCG4fUIR1w8v4P33PE35fBTk2Glhrl7nLcwGCnM971Jv29Gfb7YcB+DATyfJzbSTn+Ook05thAU3IafkeNXnvJIyWjZtckZc1zZmPnjkdl6bNoZ24a6pXJ71BCs+/y9bnrqTz56dyvGKSnYcyPBaOzQyhHxLYdXnAksRoZHNz4gbMKY3S/ct4tkNjzJ/yhtn1A9KGkDKmu1e655OQ5/4bSmlzAZwv56a4EcCx06Ls7jLIt3va5afgRBiqhBitxBid0HhmUfC2iZYQnh+njW9KcUlTvoMzuTNZaV062zCoBc4NNiz18adk4LYuTWCgMaC+Yut3vX4D6B93/RArFbJoMQ83l5WRpfORgx6V9AnH7cgZUsYye81Z9k7ZXy7s9J77VrKamrffU8AVqtk2JACli8vp1NnAwa3N/x7bQgbN4ey8t1mrFxRznc7bV5r10ZN7XHTQiizOpk2Io11K4qJTfBHXw8nFEQtZTVn+PsteQybs5RbX3yP1V/tYeGdowAIbOTHNV3acv1Ty0ic/TaNTEZG9OrgvXYt4rWdXdj+8fdMSbifJ2+cx+Snx3nUGYwG+o3qxZcf7vBa16P9BbXyPbVul3OUn1ko5RJgCUDPbn5nxESa9WRmVR+lMrMdmMM9px1BgTqWLAo99X107GMhJtpA+UknkWY9vXu6phQ3jgyo0w+9IbUjzDqysqpNNytbI7ylp3ZgoI5XFzar0u7ZN5fW0a6YU8vZIlTP9cMb8cMeG/37+uENZrPOY8SWnaMRFu55nAsM1LHgpaZV2gP65dOqlUsz3K0dGqpn6DA/9uyx06evCW8IDTeSn129zvOzHYSEGT1iAgL1zPqnuUp70p+OEB7lGXMh5JaUER5cfXI9LLgJeaWeo8ATFdWG+c2+ozw2VkdwgD9XxrUis7CU4jLXyHLbT6l0axPBJ7sPeKWdbymiRVT1yCU0KoTCrJoTimr2fr0fc7twgpoHUlroGn1dObw7qT+kUZLn/X52Og09ksl1T4Fwv+a5yy1Aq9PiooAsd3lULeV15orufqSmOTiaYcdmk3y07gQjhjT2iCmxathsLn9avqqMgX39CQrUER5mICrCwKFU13A75euTdIzzfmdsSO0e3U38luYgPcOBzSZZu66cYUM8r0xZrc4q7XdXldOvj4nAQB0nyp0cL3NNXU6UO/niy0o6xnuv3a2bkbQ0jQy39n/WVZCY6GlQp2uvXnWS3m7t8nInZW7t8nInX39lIz7e+2NkfFd/Mo/ayD5mw26TfLmhlH6DPacsZaUadrf2pmQrXXo3JiCw7ie4a/JrRg7RLZoRGRKEQa9jWM94vtz7m0dM88Dq7d85uiU6ISg5UUFO8XG6xpjxN7r62qd9NGm5ZzeJmhzclUpknJnwmDAMRgODxg1gx/rdHjER7cKr3sf2aIPRZKgyGIBrkgaSsuabOvX5dBp6JLMe+DPwgvt13Wnlq4QQLwERuE7wfi+l1IQQx4UQfYHvgEnAqxcibDAIXnouhBsm5KJpMCmpCQnxJt5eWQrAXycFcfCwnTvvK0Cvgw7tTbyxoPqIsODZEO6Yno/dLomJNvDWQu8vpza09gvPBjN2QgFOJ0wYF0CHeCPLV7qOrHdMCuDQYTt331eMXi+Ib2/g5fmuUU1+vpM/T3HN7x0a3DymMddd4/2lc4NB8MwzQUy8rRjNCePGNSI+3si777rOr0yc2JjUVAcz77Oi10NcnIF585tWaU+9s6RKe8wYfwZd490ICkBvEEx/siWP/fkYTicMHduUmPZ+bHi/GICRtzUjI9XGvAez0OkFrWNNPPCiuar98/dm8vN35ViLNSb0T2XifaEMHxfslbbmlMz96HPeuPsmdDrBxzt/5UhOIWMHdAXgw+0/k9g9jlsHdsPhdFJpd/DICtdl4r3pOWzdc5g1D9+Gpjk5kJnPR9/u9brfTs3J4hlLmbv5cXR6HVuWp5C+z8LIuxIB2PDWVq66uQ+DJ16NZteoPGnj2aSFVe39Gpm4IrEri6Yt8VqzJuJiZZAUQqwGBgGhQC4wB/gY+ACIBjKAsVLKInf848BfAAdwv5Ryk7u8F9WXsDcBM853CbtnNz+pHr95sbXV4zcvNg38+M3/Sil71VZ30UYyUsrxZ6m67izxzwHP1VK+G+hcj4umUCh8SEOfk1EoFP/jKJNRKBQ+RZmMQqHwKcpkFAqFT1Emo1AofIoyGYVC4VOUySgUCp+iTEahUPgUZTIKhcKnKJNRKBQ+RZmMQqHwKcpkFAqFT1Emo1AofIoyGYVC4VOUySgUCp+iTEahUPgUZTIKhcKnKJNRKBQ+RZmMQqHwKcpkFAqFT1Emo1AofEpD511qUD5NKeehfxShOWHy+CbMmhHsUV9cojHtgULS0u34+QnefCmUTh1cGQtLrBp3zypk3wEbQgjefKk5fXp5n4OoIbW3pVTw2BNWnE7J7eMDuG96oEd9SYmTex8s5mi6Az8/wSsLmtGxgyuJW48+OTRpItDrBHoDbNsUVpvEWfkipZIn55SiaZA0vhH3TPdMsFZS4uShB62kp2v4+QnmLwgi3q3dv28eAQE69HrQG+CTjd7nmwLY9WUZbzydh9MpGXZrMEl/88wJfdyqseCRbLLT7Zj8BA+8aKZNvCu304KHs9mZUkZwcz1vb25bJ12A/h1b88hNg9DpdKzd8QvLPtvlUd8rNopFf72BzEJXlsbPf07lrc3fAXD7oB7c1K8LUkoOZxfwxPufYnN4n3Km19Du3L3oDnR6HZuWbiP5xY896vvd0IvJTychnRLNofH6zHf4dfsBotpHMHvNzKq48LZhrJiTzNqXN9ap7w1uMkKIZcBIIE9K2dldFgIkAzHAUeBWKWWxu+5RYAqgAfdKKbdciK6mSWY+VsSGNS2JNBu46vosRgxtTMf21WlP//mKla6dTCQvC+PgYRszHy9i4weubHsPPVFE4qBGrHo7DJtNUn7S+6T3Da39yOMlfLQ6lAiznsTr8xg2xJ/49tWZIBe+epzOnYysXNqcw6l2Hn7MytoPqn/QH38YSvOQumdW1DTJ7NmlvL+qGWaznlEjCkkc4k/79tW74WuvlpHQycjbS5uRmupg9uOlrEkOqapP/jCEkJC6D8A1TbJ4Ti4vrGxFaLiRGWOO0m9wE1rHVSeIW/16Ie06+vPkm1FkHKlk8RO5zHs/GoDEW5pyw6RmzJtV94SlOiF4bOy13PXa/5FbcpxVsybwxS9H+C3HMxPkj0cymbFknUdZWNMAJlzdgxufX0GlXWPeHSMY1jOe9d/v805bp2PG4ik8MuQZCixFLP5+LjvW7yZjf3VK+R+3/cKO9bMAaNMlmtnJDzAl4X4sh7KY1vOhqu9ZbXmL7Wu/r3v/69yi/nkHGFaj7O/ANillHLDN/RkhRAKQBHRyt3ldCHFBeUR3/1hJuxgDbVobMZkEt4wOYMOWco+Y/YftXDPQNUKIjzORfsxBbr5G6XEn3+ysZPIE11HYZBIEN/V+MRpS+4cfbbSJMRDT2oDJJLhxdGM2banwiDl4yM6fBrp+fHGxRo5ZHOTl//5kbXv22ImJ0dParT1qtD+ffuqpffiwxoCBLrONjTVgsWjk14P2wZ8qiGhtwhxtwmgSXD0yiG+3lnnEZByupEd/V7rY6HZ+5GbaKc535c/u2rsxgcEX9nPp3DqcY/klZBZacWhONv9wkEFd2nndXq/T4Wc0oNcJGhkN5JeWnb+Rm/jesWSl5pCTlofD7uCL5O30H+2Zg63iRPU28A/wh1pyJfa4rjPZR3LIyyjwWvsUDW4yUsqvgJrJfUcDK9zvVwBjTitfI6WslFKmAalA7wvRzcrRiIyoPoJGmg0eyeABuiSYWLfJ9ePf9WMlGRYHmdkO0tLthDbXcdfMAvomZvG3Bws4Ue79aKIhtbNznEREVJtShFlPdo6nducEIxs2uhK8//CjjWMWrWr5hIBbxhdy7bA8VrznmTT+fORkO4kwV2ubw/XkZnsue8cEA5s3uXb6PT/ayLRoZLtjhBDcPqGI64cX8P57nqZ8Pgpy7LQwV6/zFmYDhbmeGTbbdvTnmy2uHNAHfjpJbqad/BxHnXRqIyy4CTkl1bml80rKaNm0yRlxXduY+eCR23lt2hjahbumcnnWE6z4/L9seepOPnt2KscrKtlxIMNr7dDIEPIthVWfCyxFhEY2PyNuwJjeLN23iGc3PMr8KW+cUT8oaQApa7Z7rXs6DW4yZ6GllDIbwP16auIfCRw7Lc7iLjsDIcRUIcRuIcTugsIzj4S1JbYVwvPzrOlNKS5x0mdwJm8uK6VbZxMGvcChwZ69Nu6cFMTOrREENBbMX2z1unOXuvZ90wOxWiWDEvN4e1kZXTobMehdQZ983IKULWEkv9ecZe+U8e3OSu+1aymrqX33PQFYrZJhQwpYvrycTp0NGNze8O+1IWzcHMrKd5uxckU53+20ea1dGzW1x00LoczqZNqINNatKCY2wR99PZxQELWU1cysvN+Sx7A5S7n1xfdY/dUeFt45CoDARn5c06Ut1z+1jMTZb9PIZGRErw7ea9ciXltW5+0ff8+UhPt58sZ5TH56nEedwWig36hefPnhDq91PdpfUKuGo9btVVuglHIJsARcubBr1kea9WRmVR+lMrMdmMM9px1BgTqWLAo99X107GMhJtpA+UknkWY9vXu6phQ3jgyo0w+9IbUjzDqysqpNNytbI7ylp3ZgoI5XFzar0u7ZN5fW0a6YU8vZIlTP9cMb8cMeG/37epf43mzWeYzYsnM0wsI9j3OBgToWvNS0SntAv3xatXJphru1Q0P1DB3mx549dvr0NeENoeFG8rOr13l+toOQMKNHTECgnln/NFdpT/rTEcKjPGMuhNySMsKDq0+uhwU3Ia/UcxR4oqLaML/Zd5THxuoIDvDnyrhWZBaWUlzmGllu+ymVbm0i+GT3Aa+08y1FtIiqHrmERoVQmFVz4lDN3q/3Y24XTlDzQEoLXaOvK4d3J/WHNEryvN/PTudSHcnkCiHMAO7XPHe5BWh1WlwUUPczccAV3f1ITXNwNMOOzSb5aN0JRgxp7BFTYtWw2Vz+tHxVGQP7+hMUqCM8zEBUhIFDqa7hdsrXJ+kY5/3O2JDaPbqb+C3NQXqGA5tNsnZdOcOGeF6ZslqdVdrvriqnXx8TgYE6TpQ7OV7mmrqcKHfyxZeVdIz3XrtbNyNpaRoZbu3/rKsgMdHToE7XXr3qJL3d2uXlTsrc2uXlTr7+ykZ8vPfHyPiu/mQetZF9zIbdJvlyQyn9BntOWcpKNexu7U3JVrr0bkxA4AWd8vPg14wcols0IzIkCINex7Ce8Xy59zePmOaB1du/c3RLdEJQcqKCnOLjdI0x42909bVP+2jScs9uEjU5uCuVyDgz4TFhGIwGBo0bwI71uz1iItqFV72P7dEGo8lQZTAA1yQNJGXNN3Xq8+lcqiOZ9cCfgRfcr+tOK18lhHgJiADigLqf7gYMBsFLz4Vww4RcNA0mJTUhId7E2ytLAfjrpCAOHrZz530F6HXQob2JNxZUHxEWPBvCHdPzsdslMdEG3lro/eXUhtZ+4dlgxk4owOmECeMC6BBvZPlK15H1jkkBHDps5+77itHrBfHtDbw83zWqyc938ucprvm9Q4ObxzTmumu8v3RuMAieeSaIibcVozlh3LhGxMcbefdd1/mViRMbk5rqYOZ9VvR6iIszMG9+0yrtqXeWVGmPGePPoGu8G0EB6A2C6U+25LE/H8PphKFjmxLT3o8N7xcDMPK2ZmSk2pj3YBY6vaB1rIkHXjRXtX/+3kx+/q4ca7HGhP6pTLwvlOHjgr3S1pySuR99zht334ROJ/h4568cySlk7ICuAHy4/WcSu8dx68BuOJxOKu0OHlnhuky8Nz2HrXsOs+bh29A0Jwcy8/no271e99upOVk8YylzNz+OTq9jy/IU0vdZGHlXIgAb3trKVTf3YfDEq9HsGpUnbTybtLCqvV8jE1ckdmXRtCVea9ZE1DY/u5gIIVYDg4BQIBeYA3wMfABEAxnAWCllkTv+ceAvgAO4X0q56XwaPbv5ye2bI3yx+Jc05dJ+/iCfaf/+K0IXygFbcINpz1o4tcG0wxZ/22Dan8mP/iul7FVbXYOPZKSU489Sdd1Z4p8DnvPdEikUivrkUj0no1Ao/kdQJqNQKHyKMhmFQuFTlMkoFAqfokxGoVD4FGUyCoXCpyiTUSgUPkWZjEKh8CnKZBQKhU9RJqNQKHxKg99WcDGwS4lF8/65J/VJlN77m/jqm2a6xucP8hHl2vHzBykuC9RIRqFQ+BRlMgqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqdcFrcVnI2vUip47slSNA3Gjm/MXfd4Jvuyljh5dFYJx9I1TH6CufOb0r5DdTIzTZPcNKKAluF6lrwTUiftT1PKeegfRWhOmDy+CbNmBHvUF5doTHugkLR0O35+gjdfCqVTB1e2xBKrxt2zCtl3wIYQgjdfak6fXt7nP9r8+QlmPpGPpsGUCUE8MsNz2YtLNKbMzOW3dDv+foJ/LWxJ5w5+HEy1MX5adlXcb+kOnnoohPumNvNa+4uUSp6c41rnSeMbcc90z3VeUuLkoQetpKdr+PkJ5i8IIt69zvv3zSMgQIdeD3oDfLLR+3xTALu+LOONp/NwOiXDbg0m6W+eOaGPWzUWPJJNdrodk5/ggRfNtIl33Ray4OFsdqaUEdxcz9ub29ZJF6B/x9Y8ctMgdDoda3f8wrLPdnnU94qNYtFfbyCz0JWl8fOfU3lr83cA3D6oBzf164KUksPZBTzx/qfYHN6nnOk1tDt3L7oDnV7HpqXbSH7xY4/6fjf0YvLTSUinRHNovD7zHX7dfoCo9hHMXjOzKi68bRgr5iSz9uWNder7JWsyQohlwEggT0rZ2V0WAiQDMcBR4FYpZfGFfL+mSZ6aXcryVSGEm/XcPLKA6xL9iG1fbSJvLi6jYycjr/8rhCOpDp6abWXlmuodc8XSE7SLNVBWVrfcVZommflYERvWtCTSbOCq67MYMbQxHdtXp1z95ytWunYykbwsjIOHbcx8vIiNH7gy/T30RBGJgxqx6u0wbDZJ+Unn2aRq1Z7xWD5bkiOJMhvoMzyDUUMCSIivvsdq7itFdO/sx/8tj+DAYRszHstj64dRxMea+OGz1lXf06pHGmOGn5k4/lzas2eX8v6qZpjNekaNKCRxiD/t21fvhq+9WkZCJyNvL21GaqqD2Y+Xsia52gSTPwwhJKTuA3BNkyyek8sLK1sRGm5kxpij9BvchNZx1f1e/Xoh7Tr68+SbUWQcqWTxE7nMez8agMRbmnLDpGbMm1X3hKU6IXhs7LXc9dr/kVtynFWzJvDFL0f4LcczE+SPRzKZsWSdR1lY0wAmXN2DG59fQaVdY94dIxjWM5713+/zTlunY8biKTwy5BkKLEUs/n4uO9bvJmO/pVp32y/sWD8LgDZdopmd/ABTEu7HciiLaT0fqvqe1Za32L627rkUL+Xp0jvAsBplfwe2SSnjgG3uzxfEz3vstI7RE93agMkkGHFDIz771PMmytTDDvoNcO2E7WINZB7TKMh3HUFysjW++LySsePrfhPi7h8raRdjoE1rIyaT4JbRAWzYUu4Rs/+wnWsGukYn8XEm0o85yM3XKD3u5JudlUye4Ppxm0yC4Kbep1L9/scK2sUYaevWHjc6kPVbPPMy7ztk49qBrn51iDNx9JiD3HyHR8y2r8tpF2OkdSvv09Tu2WMnJkZPa/c6HzXan08/rfCIOXxYY8BAl9nGxhqwWDTy839/oriDP1UQ0dqEOdqE0SS4emQQ324t84jJOFxJj/6ufke38yM3006xu99dezcmMPjCfi6dW4dzLL+EzEIrDs3J5h8OMqhLO6/b63U6/IwG9DpBI6OB/NKy8zdyE987lqzUHHLS8nDYHXyRvJ3+oz1zsFWcqN4G/gH+UEvCxx7XdSb7SA55GQVea5/ikjUZKeVXQM2kv6OBFe73K4AxF/r9uTka4RHVP85ws47cHM+duUNHA59ucm2An360kZWpkZPtGjU892QpDz8WhO4C1mBWjkZkRPXRO9Js8EhED9AlwcS6TS7j2fVjJRkWB5nZDtLS7YQ213HXzAL6JmbxtwcLOFHu/UgmM8dBq0hP7cwcTwPpluDH2o2uHfn7HytIt9ixZHnGJK8rI2lMIHUhJ9tJhLl6nZvD9eRmey57xwQDm93rfM+PNjItGtnuGCEEt08o4vrhBbz/nqcpn4+CHDstzNX9bmE2UJjrmWGzbUd/vtniunv8wE8nyc20k19j3VwIYcFNyCmpvis9r6SMlk3PHAF2bWPmg0du57VpY2gX7hox51lPsOLz/7LlqTv57NmpHK+oZMeBDK+1QyNDyLcUVn0usBQRGtn8jLgBY3qzdN8int3wKPOnvHFG/aCkAaSs2e617ulcsiZzFlpKKbMB3K9hZwsUQkwVQuwWQuwuKjrzR1hbdl4hPD/fdU8TSq1Obhiaz7vvlNOxkxG9AVI+q6B5cx2du3p/FK+r9qzpTSkucdJncCZvLiulW2cTBr3AocGevTbunBTEzq0RBDQWzF9srVftR2Y0o9iq0XNwOouXltCjsx8GQ3WQzSb5z5Yybhnl/VQJoLZJZU3tu+8JwGqVDBtSwPLl5XTqbMDg9oZ/rw1h4+ZQVr7bjJUryvlup61O+ufTHjcthDKrk2kj0li3opjYBH/09XBCQdRSVjM99H5LHsPmLOXWF99j9Vd7WHjnKAACG/lxTZe2XP/UMhJnv00jk5ERvTp4r12LeG2pqbd//D1TEu7nyRvnMfnpcR51BqOBfqN68eWHO7zW9Wh/Qa3+AEgplwBLALp0NZ2xVsPNenKyqkcPOdlOwlp6TjuaBOp44aXgU9/Htf3zadVKzyfrT7JtawVfplRSWSkpO+5k1r3FzH/FuxOgkWY9maeNDDKzHZjDPbWDAnUsWRRapd2xj4WYaAPlJ51EmvX07umaxt04MqBOJhNlNnAs01M7oqXnbhAUqGfZovAq7Xa9j9Imujpm0+cn6NHFn5Yt6rb7mM06jxFbdo5GWLjncS4wUMeCl5pWaQ/o51rnAOHudRQaqmfoMD/27LHTp68JbwgNN5KfXd3v/GwHIWGeB4mAQD2z/mmu0p70pyOER13YgeR0ckvKCA+uHvWFBTchr9Rzinqiotowv9l3lMfG6ggO8OfKuFZkFpZSXHYSgG0/pdKtTQSf7D7glXa+pYgWUdUjl9CoEAqzak4Qqtn79X7M7cIJah5IaaFr9HXl8O6k/pBGSZ73+9np/NFGMrlCCDOA+zXvQr+oSzcjR49qHMtwYLNJPll/kusSPR8wVWp1YrO5/OmD1Sfp1cdEk0Ads/4exNe7WpKyI4yFrwXTd4Cf1wYDcEV3P1LTHBzNsGOzST5ad4IRQzzP7ZRYtSrt5avKGNjXn6BAHeFhBqIiDBxKdQ31U74+Scc4738IV3b3JzXNRppbO3ndcUYNDTir9r/eL+Wqvo0ICqw2wTUfHyfpxrqNYgC6dTOSlqaR4V7n/1lXQWKNdW49bZ2vXnWS3n1MBAbqKC93UlbmGpGWlzv5+isb8fHem1x8V38yj9rIPmbDbpN8uaGUfoM9+1BWqmF3a29KttKld2MCAr0/33U2fs3IIbpFMyJDgjDodQzrGc+Xe3/ziGkeWL39O0e3RCcEJScqyCk+TtcYM/5GV1/7tI8mLffsJlGTg7tSiYwzEx4ThsFoYNC4AexYv9sjJqJdeNX72B5tMJoMVQYDcE3SQFLWfFOnPp/OH20ksx74M/CC+3XducPPjsEgeOKZIKbcXoSmwS3jGhEXb2T1u64jzPiJARxJdfDw/SXo9BAbZ+D5fwbXRx8wGAQvPRfCDRNy0TSYlNSEhHgTb68sBeCvk4I4eNjOnfcVoNdBh/Ym3lhQfTRa8GwId0zPx26XxEQbeGuh95dyDQbBK8+HMXx8JpoGdyQF0SnejzdXlAAw7c/B7D9sY/K9ueh10LG9iX+91LKqfXm5k8++KufNeWedqZ5T+5lngph4WzGaE8aNa0R8vJF333WdX5k4sTGpqQ5m3mdFr4e4OAPz5rtGNfn5Tqbe6VpGhwZjxvgz6BrvnzqoNwimP9mSx/58DKcTho5tSkx7Pza877o4OfK2ZmSk2pj3YBY6vaB1rIkHXjRXtX/+3kx+/q4ca7HGhP6pTLwvlOHjgr3S1pySuR99zht334ROJ/h4568cySlk7ICuAHy4/WcSu8dx68BuOJxOKu0OHlnhuky8Nz2HrXsOs+bh29A0Jwcy8/no271e99upOVk8YylzNz+OTq9jy/IU0vdZGHlXIgAb3trKVTf3YfDEq9HsGpUnbTybtLCqvV8jE1ckdmXRtCVea9ZE1DY/uxQQQqwGBgGhQC4wB/gY+ACIBjKAsVLK89p6l64m+X91/J+K+qIhH7/pJ37/UP9CyWzAx28esAU3mPashVMbTDts8bcNpv2Z/Oi/UspetdVdsiMZKeX4s1Rdd1EXRKFQ/C7+aOdkFArFHwxlMgqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqdcsrcV1CeHSsMYvPm+BtH+bNjC8wf5iKjffwPxH5IwvfdPjqtvgo/Yzx90maFGMgqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqcok1EoFD7lsrit4GxcHRXDnL7XoReCNQd/5o2fv/eov6vLlYyOTQDAIASxwc3p8f5rWCsrztv2fHyVUsFzT5aiaTB2fGPuuscz0Zi1xMmjs0o4lq5h8hPMnd+U9h2qU5xomuSmEQW0DNez5J2QOml/mlLOQ/8oQnPC5PFNmDUj2KO+uERj2gOFpKXb8fMTvPlSKJ06uDI1llg17p5VyL4DNoQQvPlSc/r08vda+4uUSp6c4+p30vhG3DPds98lJU4eetBKerqGn59g/oIg4t397t83j4AAHXo96A3wSR3T3Gz/ooJ5T5Xg1CQ3JgXwl7uDPOpLrU7mPFSEJV3D5AdP/TOE2HgjlRWSv9yah90GDodk8PWNuPuBpnXS7n1FG2ZMvQ6dTvDJpz+z6sPvPOq7d2nFc/+4iezcEgC+/vYwK1Z/61Xb89FraHfuXnQHOr2OTUu3kfzixx71/W7oxeSnk5BOiebQeH3mO/y6/QBR7SOYvWZmVVx42zBWzElm7csb66T/hzQZIcQyYCSQJ6XsfCHfoROCZ/onctumD8g5cZz1oyfyWcYRDpdUJyd/a+8u3tq7C4DrottxZ+crsFZWeNX2XGia5KnZpSxfFUK4Wc/NIwu4LtGP2PbVJvLm4jI6djLy+r9COJLq4KnZVlauqU7wtmLpCdrFGigrq1veLE2TzHysiA1rWhJpNnDV9VmMGNqYju2r073+8xUrXTuZSF4WxsHDNmY+XsTGD1xZBh96oojEQY1Y9XYYNpuk/OSZecbPpT17dinvr2qG2axn1IhCEof407599W742qtlJHQy8vbSZqSmOpj9eClrkqtNNPnDEEJC6j4A1zTJ3H8U8+b7LWgZrue2G/K4enAj2p22zv+1uJT4BBMLlwSRlmpn7j9KWLK6BSY/eHt1CxoH6LDbJXfcksfAQf507eldTi2dTnD/3wbz4OwPyC84zlsLJ7F9Zyrpxzz3l59/tfDoU/++oLZn19YxY/EUHhnyDAWWIhZ/P5cd63eTsd9SFfPjtl/YsX4WAG26RDM7+QGmJNyP5VAW03o+VPU9qy1vsX1t3Q6mcJ7pkhAiVggxoJbyq4QQ7eqsVn+8Awz7PV/QvYWZo6XFHDtuxe508p/fDpDYOvas8aPbdmDdkQMX1LYmP++x0zpGT3RrAyaTYMQNjfjs00qPmNTDDvoNcO3E7WINZB7TKMh35ZHOydb44vNKxo5vfMZ3n4/dP1bSLsZAm9ZGTCbBLaMD2LCl3CNm/2E71wx0jU7i40ykH3OQm69RetzJNzsrmTzBNfowmQTBTb2/C3PPHjsxMXpau/s9arQ/n35a4RFz+LDGgIEuw4uNNWCxaOTna7V9XZ34ZY+NVjEGoqINGE2CoaMa8cXWkx4xvx120Me9ztvEGsmyOCjM1xBC0DjA9VNxOCQOe+2J7M9Gx/ZmMrNKyM6x4nA4+fyr/Qzs693+8nvaAsT3jiUrNYectDwcdgdfJG+n/2jPHGwVJ6q3gX+AP9SS8LHHdZ3JPpJDXkaB19qnON8hYRFQWyrAk+66BkFK+RXgfULgWghv3ITsE9Vdyz5xnPDGted39tcbuDqqDZuOHqpz29rIzdEIj6j+cYabdeTmeP6QOnQ08Okm18b/6UcbWZkaOdmuUcNzT5by8GNB6C7gjFpWjkZkRPXIIdJsICvbU7tLgol1m1zGs+vHSjIsDjKzHaSl2wltruOumQX0Tczibw8WcKLc+5FMTraTCHN1v83henKzPdt3TDCw2d3vPT/ayLRoZLtjhBDcPqGI64cX8P57nsZ4PvJyNMJP025p1pNXY523TzCybZPLePbusZGdqVVtF02T3Do8l2t7ZtP3Kj+69PA+M2ho8ybkFVTvL/kFxwltHnhGXKcOESx9dTLznrqFmOjmdWp7Vu3IEPIt1aOeAksRoZHNz4gbMKY3S/ct4tkNjzJ/yhtn1A9KGkDKmu1e657O+XbTGCnlzzULpZS7gZgLUrxICCGmCiF2CyF2a8dP1BJwZtHZJh6DW7djd14m1sqKOretjdoyA9c8Mt51TxNKrU5uGJrPu++U07GTEb0BUj6roHlzHZ27XlgKWm+0Z01vSnGJkz6DM3lzWSndOpsw6AUODfbstXHnpCB2bo0goLFg/mKr99q1lNXUvvueAKxWybAhBSxfXk6nzgYMbk/899oQNm4OZeW7zVi5opzvdtrqVfsvfwuktNTJrcNzWfNOGfGdjOjdvqTXCz7Y1JItO838ssdO6kHvH+kgah32eC7RodRcxt3xJlNmvMO///MDz82+yeu259aupXUtO8H2j79nSsL9PHnjPCY/Pc6jzmA00G9UL778cIfXuh7tz1N/rjN6jS5I8SIhpVwCLAHwi4k6Y63mnCjDHFB9RDAHBJJbXvtzSEa17ch691Sprm1rI9ysJyer+iiak+0krKXntKNJoI4XXgo+1Reu7Z9Pq1Z6Pll/km1bK/gypZLKSknZcSez7i1m/ivNvNKONOvJzHJUfc7MdmAO99QOCtSxZFFolXbHPhZiog2Un3QSadbT230u4saRAXUyGbNZ5zFqys7RCAv3PM4FBupY8FLTKu0B/Vz9Bgh3L2doqJ6hw/zYs8dOn74mvKFluJ6c07RzszVa1LLOn54fUqV9/cAcIlt5/kSCmuro1c+P7V9UEBvvndHnFxwnLLR6f2kRGkhBoef+Un6y2jC/2/0b+rsTaRrUyKu259S2FNEiqnrkEhoVQmHW2ScBe7/ej7ldOEHNAyktdI2grhzendQf0ijJ835bn875RjK7hBB/rVkohJgC/PeCFC8RfsrPpk1QM1o1aYpRp2NU2w5sTU89Iy7QaKJveBSfnlbnbduz0aWbkaNHNY5lOLDZJJ+sP8l1iZ7D71KrE5vN5Y0frD5Jrz4mmgTqmPX3IL7e1ZKUHWEsfC2YvgP8vDYYgCu6+5Ga5uBohh2bTfLRuhOMGOJ5bqfEqlVpL19VxsC+/gQF6ggPMxAVYeBQqusonvL1STrGeT+i6tbNSFqaRoa73/9ZV0FijX5bT+v36lUn6d3HRGCgjvJyJ2VlrmlTebmTr7+yER/v/XWLTt1MZKQ5yMxwYLdJtvznJFcneh4nS61O7G7t/1tzgit6+9EkUEdRoUap1aVdUSH57psK2sR6r33gUDZRkc0Ib9kUg0HHtX/qyPbvPPeXkGYBVe87tA9HJwTW0pNetT0XB3elEhlnJjwmDIPRwKBxA9ixfrdHTES78Kr3sT3aYDQZqgwG4JqkgaSs+cZrzZqcb03dD6wVQtxGtan0AkzAjResegmgSckT337GyuG3oBc6Pji0l8MlhdzWoRsA7x/4CYChMXF8lXmUkw77edt6i8EgeOKZIKbcXoSmwS3jGhEXb2T1u65p3fiJARxJdfDw/SXo9BAbZ+D5fwbXS78NBsFLz4Vww4RcNA0mJTUhId7E2ytLAfjrpCAOHrZz530F6HXQob2JNxZUHwkXPBvCHdPzsdslMdEG3lro/WVkg0HwzDNBTLytGM0J48Y1Ij7eyLvvus6vTJzYmNRUBzPvs6LXQ1ycgXnzXaOa/HwnU+8sAcChwZgx/gy6xvvzIgaD4O9PB/O3SQU4NcnoWwOIbW/kw/dco4KxtzchLdXO7AeK0euhbayBJ//pGtUU5Gn844FinE5wOiVDRjbmT9d5P5DXnJJFb3zG/GfGotMJNm7dy9GMQm4Y3h2A9Zv2cPWA9oy+vgea5qTS5uCpeevP2dZbnJqTxTOWMnfz4+j0OrYsTyF9n4WRdyUCsOGtrVx1cx8GT7waza5RedLGs0nVT3P0a2TiisSuLJq2xGvNmoja5mdnBAlxDXDqUvGvUsrPL1ixHhBCrAYGAaFALjBHSrn0bPF+MVEyfPbl+PhN73+E9U2Bs+L8QT6iSGu4/8y4928zGkzbtGlXg2l/Jj/6r5SyV211Xm0NKWUKkFKvS/U7kFKOb+hlUCgU3qFuK1AoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqcok1EoFD5FmYxCofApf8jHb9YVv/Ry2k+t+2MD64PBS2aeP8hHvHLt+w2mPdDf++et1Dd/ienbYNomGu7+oUsVNZJRKBQ+RZmMQqHwKcpkFAqFT1Emo1AofIoyGYVC4VOUySgUCp+iTEahUPgUZTIKhcKnKJNRKBQ+RZmMQqHwKZfFbQVno9fQ7ty96A50eh2blm4j+cWPPer73dCLyU8nIZ0SzaHx+sx3+HX7AaLaRzB7TfXtAuFtw1gxJ5m1L2/0WvvqqBjm9L0OvRCsOfgzb/zsedvDXV2uZHRsAgAGIYgNbk6P91/DWllx3rbn44cvS/nXM1k4NUniuBBuntbSo77M6uDVR46Rk2HD5CeY/kIrWsc38qrt+diWUsFjT1hxOiW3jw/gvumeeZ1LSpzc+2AxR9Md+PkJXlnQjI4dXAnkrFYn988qZv9BB0LAKwuCubKX92lfGnJ7X67a4GXepYuJEGIY8DKgB/4lpXyhRn0HYDnQE3hcSjn/fN8ZJEJkH3GdR5lOp2P5wZd5ZMgzFFiKWPz9XJ6f8DIZ+y1VMf4B/lSccOUPatMlmtnJDzAl4f4zvme15S1m9H2UvIyCM7QPLel9RplOCL4Yeye3bfqAnBPHWT96IvembDhrgrjrottxZ+crGL/xgzq1re3eJU2T3D34AE+taEvzcCMP3XiYBxe1plVcdUbid+Zm4R+gI+necCxHKnhrTibPvNfOq7anGOh/5rrQNEmfq3L5aHUoEWY9idfnseT1EOLbV2ehnPOMlYAAwcMPBHE41c7Dj1lZ+4Ergdw99xXRt48fEycEYLNJTp6UNG165mA8Karfmev8Im3v2rgctM+Vd+mSmi4JIfTAa8BwIAEYL4RIqBFWBNwLnNdczkV871iyUnPIScvDYXfwRfJ2+o/2XEenVjy4NkRt2ep7XNeZ7CM5Xm90gO4tzBwtLebYcSt2p5P//HaAxNaxZ40f3bYD69y5uOvatiaHfyrH3NpEeLQfRpOOgSOD+e4zzxzHx1Ir6Nq/CQBR7fzJy7RRUmD3qu25+OFHG21iDMS0NmAyCW4c3ZhNWzyTwB08ZOdPA12jk7hYI8csDvLyNY4fd7LjOxu3j3el1DWZRK0GczYacntfrtqnuKRMBugNpEopf5NS2oA1wOjTA6SUeVLKXcDvus03NDKEfEv10b/AUkRoZPMz4gaM6c3SfYt4dsOjzJ/yxhn1g5IGkLJme520wxs3IftEda7h7BPHCW/cpNZYf72Bq6PasOnooTq3rY2iXDuh5uok9c3DjRTleq7KmI6N2LnFZR6HfionP9NGQbbdq7bnIjvHSUREdZL7CLOe7BzNI6ZzgpENG08CLlM6ZtHIytY4mu6geXMdM2aWcM2QPO6bVcyJcqfX2g25vS9X7VNcaiYTCRw77bPFXVZnhBBThRC7hRC77VTWUn9mm9qmjts//p4pCffz5I3zmPz0OI86g9FAv1G9+PLDHXVcuFq0zxI6uHU7dudlYq2sqHPb2vBmdnzzXWGUWTXuH3mQT1YW0DahEXqD8KptXbVrbof7pgditUoGJebx9rIyunQ2YtALHBr8vNfOHZMCSPk0jIDGglcWl3mt3ZDb+3LVrmp/Qa18Ry2rpE6/oepGUi4BloDrnEzN+nxLES2iqh09NCqEwqyis37f3q/3Y24XTlDzQEoLXSOJK4d3J/WHNEryvJ8yAOScKMMcUH3C0xwQSG557T+YUW07st49Vapr29poHm6kINtW9bkwx05IS6NHTONAPffOiwZcO+TUq/fTMspE5Unnedueiwizjqys6pFLVrZGeEu9R0xgoI5XFzar0u7ZN5fW0XrKT0oizHqu6OkaSY0a0YiXFx/HWxpye1+u2qe41EYyFqDVaZ+jgCxfCB3clUpknJnwmDAMRgODxg1gx/rdHjER7cKr3sf2aIPRZKha8QDXJA0kZc03ddb+KT+bNkHNaNWkKUadjlFtO7A1PfWMuECjib7hUXx6Wp23bc9GXNfGZB+1kXusErvNyTcbSuh9XVOPmLJSDbvNNRXZmlxEpyub0DhQ71Xbc9Gju4nf0hykZziw2SRr15UzbIjnSWOr1YnN5jomvLuqnH59TAQG6mgZpicyQs/hVNf07KtvKj1OGJ+Phtzel6v2KS61kcwuIE4I0QbIBJKACb4QcmpOFs9YytzNj6PT69iyPIX0fRZG3pUIwIa3tnLVzX0YPPFqNLtG5UkbzyYtrGrv18jEFYldWTRtSZ21NSl54tvPWDn8FvRCxweH9nK4pJDbOnQD4P0DPwEwNCaOrzKPctJhP29bb9EbBH+dE8lTk39Dc8LgW0KIbu/P5lWuE3rDJoRiSa3g5VkZ6PSCVrH+TH8h6pxtvcVgELzwbDBjJxTgdMKEcQF0iDeyfOUJAO6YFMChw3buvq8YvV4Q397Ay/ObVbWf+0xTps0oxm6XtI428OpLzc4mdQYNub0vV+1TXIqXsK8HFuG6hL1MSvmcEGIagJTyTSFEOLAbCAKcQBmQIKUsPdt31nYJ+2JR2yXsi0XDPn6z7lch6ovaLmErfMu5LmFfaiMZpJQbgY01yt487X0OrmmUQqH4A3CpnZNRKBT/YyiTUSgUPkWZjEKh8CnKZBQKhU9RJqNQKHyKMhmFQuFTlMkoFAqfokxGoVD4FGUyCoXCpyiTUSgUPuWSu3fJFzTkvUsKxeXAH+bxmwqF4n8PZTIKhcKnKJNRKBQ+RZmMQqHwKcpkFAqFT1Emo1AofIoyGYVC4VOUySgUCp+iTEahUPgUZTIKhcKnXHLZCi4mvYZ25+5Fd6DT69i0dBvJL37sUd/vhl5MfjoJ6ZRoDo3XZ77Dr9sPENU+gtlrZlbFhbcNY8WcZNa+vBFvUdpK+3LQhga6d0kIMQx4GVdupX9JKV+oUS/c9dcD5cBkKeUP7rplwEggT0rZ2Ru92u5d0ul0LD/4Mo8MeYYCSxGLv5/L8xNeJmO/pSrGP8CfihOuHNRtukQzO/kBpiTcf8b3rLa8xYy+j5KX4V2uIaWttP/XtC+pe5eEEHrgNWA4kACMF0Ik1AgbDsS5/6YCb5xW9w4w7PcuR3zvWLJSc8hJy8Nhd/BF8nb6j/ZcR6dWPLg2RG0Z43tc15nsIzleb3SlrbQvF+1TNMR0qTeQKqX8DUAIsQYYDew7LWY0sFK6hlk7hRDBQgizlDJbSvmVECLm9y5EaGQI+Zbq9K4FliI69Ik7I27AmN785fkJBIc1ZfbIuWfUD0oaQMqa7UpbaSvts9AQJ34jgWOnfba4y+oac06EEFOFELuFELvtVNZSf2ab2qaO2z/+nikJ9/PkjfOY/PQ4jzqD0UC/Ub348sMddVk0pa20LwvtUzSEydTSbWr22puYcyKlXCKl7CWl7GXE74z6fEsRLaKaV30OjQqhMKvorN+39+v9mNuFE9Q8sKrsyuHdSf0hjZI8a10WTWkr7ctC+xQNYTIWoNVpn6OArAuI+V0c3JVKZJyZ8JgwDEYDg8YNYMf63R4xEe3Cq97H9miD0WSgtPB4Vdk1SQNJWfON0lbaSvscNMQ5mV1AnBCiDZAJJAETasSsB6a7z9f0AaxSyuz6XAin5mTxjKXM3fw4Or2OLctTSN9nYeRdiQBseGsrV93ch8ETr0aza1SetPFs0sKq9n6NTFyR2JVF05YobaWttM9BQ13Cvh5YhOsS9jIp5XNCiGkAUso33ZewF+O6ilQO3CGl3O1uuxoYBIQCucAcKeXSc+mpx28qFL7lXJew1TN+FQrF7+aS+j8ZhUJxeaFMRqFQ+BRlMgqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqcok1EoFD5FmYxCofApymQUCoVPUSajUCh8ijIZhULhU5TJKBQKn6JMRqFQ+BRlMgqFwqcok1EoFD5FmYxCofApDZF36ZKh19Du3L3oDnR6HZuWbiP5xY896vvd0IvJTychnRLNofH6zHf4dfsBotpHMHvNzKq48LZhrJiTzNqXNyptpa20a+DTlChCiGHAy7jyK/1LSvlCjXrhrr8eV36lyVLKH87VVggxFngS6Aj0PpWP6VzUlhJFp9Ox/ODLPDLkGQosRSz+fi7PT3iZjP2Wqhj/AH8qTlQA0KZLNLOTH2BKwv1nfM9qy1vM6PsoeRkFXq0Xpa20/9e0GyQlihBCD7wGDAcSgPFCiIQaYcOBOPffVOANL9r+AtwEfPV7li++dyxZqTnkpOXhsDv4Ink7/Ud7rqNTKx5cG4JaDLnHdZ3JPpLj9UZX2kr7ctE+hS+nS72BVCnlbwDulLOjgX2nxYwGVkrXcGqnECJYCGEGYs7WVkq53132uxYuNDKEfEth1ecCSxEd+sSdETdgTG/+8vwEgsOaMnvk3DPqByUNIGXNdqWttJX2WfDlid9I4Nhpny3uMm9ivGl7ToQQU4UQu4UQu+1U1lJ/Zpvapo7bP/6eKQn38+SN85j89DiPOoPRQL9Rvfjywx11WTSlrbQvC+1T+NJkahtq1Ozd2WK8aXtOpJRLpJS9pJS9jPidUZ9vKaJFVPOqz6FRIRRmFZ31+/Z+vR9zu3CCmgdWlV05vDupP6RRkmety6IpbaV9WWifwpcmYwFanfY5CsjyMsabtr+Lg7tSiYwzEx4ThsFoYNC4AexY73kOOaJdeNX72B5tMJoMlBYeryq7JmkgKWu+UdpKW2mfA1+ek9kFxAkh2gCZQBIwoUbMemC6+5xLH8AqpcwWQuR70fZ34dScLJ6xlLmbH0en17FleQrp+yyMvCsRgA1vbeWqm/sweOLVaHaNypM2nk1aWNXer5GJKxK7smjaEqWttJX2OfD1JezrgUW4LkMvk1I+J4SYBiClfNN9CXsxMAzXJew7Tl2Srq2tu/xG4FWgBVAC7JFSDj3XctR2CVuhUNQf57qE7VOTuVRQJqNQ+JYG+T8ZhUKhAGUyCoXCxyiTUSgUPkWZjEKh8CnKZBQKhU9RJqNQKHyKMhmFQuFTlMkoFAqfokxGoVD4FGUyCoXCp1wWtxW4b7hMv8DmoUDdHwdWfzSkvtJW2t7SWkrZoraKy8Jkfg9CiN1nuyfjf11faSvt+kBNlxQKhU9RJqNQKHyKMpnzc+FP6/nj6yttpf27UedkFAqFT1EjGYVC4VOUySgUCp9y2ZmMEGKYEOKgECJVCPH3WuqFEOIVd/3PQoie52srhBgrhPhVCOEUQnh1CfB3LscyIUSeEOKXuq+BOi9HByHEDiFEpRBi1u/V82J56q1vF6IlhAgRQmwVQhx2vzZrKD0hxKPu7XJQCHHO51j7Uk8IcYUQYq+77hVR18yKUsrL5g/XQ8mPAG0BE/ATkFAj5npgE67cT32B787XFlde7njgC6CXL5fDXfcnoCfwy0VYH2HAlcBzwKyLsI3qpW8XqgXMA/7ufv934MWG0MOVnvknwA9o495O+obQA74H+rn3xU3A8Losx+U2kqlKnSultAGn0t+eTlXqXCnlTuBU6tyztpVS7pdSHrxIy4GU8ivg7Bm66nE5pJR5UspdgL0e9M5LPfbtQrVGAyvc71cAYxpIbzSwRkpZKaVMA1Jxba+Lqufe54KklDuky3FWUsd1crmZTIOmzq2n5ahPLobGH42WUspsAPdrWAPp+Wrb1FUv0v3+gpfjcjOZBk2dW0/LUZ9cDA3FhXGxt43P9vvLzWQuldS5v2c56pOLofFHI/fUtNT9mtdAer7aNnXVs7jfX/ByXG4mU5U6VwhhwpX+dn2NmPXAJPfVnb64U+d62fZiLEd9Up99+l9hPfBn9/s/A+saSG89kCSE8BOudM1xuE7AXlQ99z53XAjR131VaRJ1XSe+PoN/qf3humpzCNfZ88fdZdOAae73AnjNXb+X064W1dbWXX4jLsevBHKBLT5ejtVANq6TsRZgig/XR7hboxRXWmALrhOBvto+9da3C9ECmgPbgMPu15CG0gMed2+Xg9Txik596gG9gF/cdYtx3yng7Z+6rUChUPiUy226pFAoLjLKZBQKhU9RJqNQKHyKMhmFQuFTlMkoFAqfokxGoVD4FGUyCoXCpxgaegEU/1sIIf4B3IbrZrsC4L+AFZiK63ESqcBEKWW5EOId4CTQAWgN3IHrv1D74Xq0xWT3d5bh+sfEwUAx8BiuRxZEA/dLKdcLIWKAd4EA96JMl1J+6+PuKrxAjWQU9Yb7gV03Az2Am3D9pyjA/0kpr5RSdgP24/rP01M0A64FZgL/ARYCnYAuQoju7pgA4Asp5RXAceBZIBHXf1o/7Y7JAxKllD2BccArvuijou6okYyiPhkIrJNSngQQQvzHXd5ZCPEsEAw0Abac1uY/UkophNgL5Eop97rb/grEAHsAG7DZHb8XqJRS2t1tYtzlRmCx25g0oL0P+qe4AJTJKOqTsz2W8R1gjJTyJyHEZGDQaXWV7lfnae9PfT61f9pl9f0vVXFSSqcQ4lTMTFz3jXXDNUKvuOBeKOoVNV1S1CffAKOEEP5CiCbACHd5IJAthDDiOl/jC5oC2VJKJzAR16NFFZcAaiSjqDeklLuEEOtxPSs2HdiN66TvP4Dv3GV7cZlOffM68G8hxFggBTjhAw3FBaDuwlbUK0KIJlLKMiFEY+ArYKqU8oeGXi5Fw6FGMor6ZokQIgHwB1Yog1GokYxCofAp6sSvQqHwKcpkFAqFT1Emo1AofIoyGYVC4VOUySgUCp/y/0UfePK4/7g1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "scores = np.array(results.mean_test_score).reshape(7, 7)\n",
    "\n",
    "mglearn.tools.heatmap(scores, xlabel='gamma', xticklabels=param_grid['gamma'], ylabel='C', yticklabels=param_grid['C'], cmap=\"viridis\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brief-window",
   "metadata": {},
   "source": [
    "And finally see some cases when the ranges of the parameters are not ideal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "certified-journal",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T16:04:26.912215Z",
     "start_time": "2021-03-21T16:04:25.239250Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsAAAAEeCAYAAACE86MtAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACD0UlEQVR4nOzdeXhTVf748fdJ0oVuQCl0ZS8UCgJqAREcUTZhQFxHwK/bgMqIjCKOzCDu+6iIDiL6AwRXUEeFURRRUYHBARQUZS1bKd0LXaClaW7O74+EtumWFNqkpZ/X8+Rpk3vO/dyT3HPuycm59yqtNUIIIYQQQjQXJl9vgBBCCCGEEN4kHWAhhBBCCNGsSAdYCCGEEEI0K9IBFkIIIYQQzYp0gIUQQgghRLMiHWAhhBBCCNGsWHy9AUIIIYQQoukbdVmwzj1m1CnPT7+WrNFaX9FAm1Qj6QALIYQQQoizlnvMYPOaDnXKY47eF9FAm1Mr6QALIYQQQoizpgE7dl9vhkekAyyEEEIIIeqBxtDSARZCCCGEEM2EYwRY+3ozPCIdYCGEEEIIUS9kCoQQQgghhGg2NBpDywiwEEIIIYRoRmQKhBBCCCGEaDY0YEgHWAghhBBCNCcyAiyEEEIIIZoNDTIHWAghhBBCNC9N4xoQ0gEWQgghhBD1QKNlDrAQQgghhGhGNBhNo/8rHWAhhBBCCHH2HHeCaxqkAyyEEEIIIeqBwkD5eiM8Ih1gIYQQQghx1jRglykQQgghhBCiOWkqI8AmX2+AEEIIIYQQ3iQjwEIIIYQQ4qw5boXcNEaApQMshBBCCCHqhV1LB1gIIYQQQjQTMgIshBBCCCGaFY3CaCKnl0kHWAghhBBC1AuZAiGEEEIIIZqNpjQFommMUwshhBBCiEZOYWhTnR4erVWpK5RSe5RSyUqpv1ezvLVS6hOl1K9Kqc1Kqd7u1ikdYCGEEEIIcdY0YMdUp4c7Sikz8CowGkgEJiqlEislmw1s11r3AW4GXna3XukACyGEEEKIemGg6vTwwAAgWWt9QGttBZYD4yulSQS+AdBa7wY6KaUia1updICFEEIIIcRZ07pBpkDEAkcqPE91vlbRL8A1AEqpAUBHIK62lcpJcEIIIYQQol7Y634SXIRSamuF529ord+o8Ly6FepKz58FXlZKbQd2ANsAW21BpQMshBBCCCHOmuMqEHWeXJCjtU6qZXkq0L7C8zggzSWu1gXAbQBKKQUcdD5qJB1gIYQQQghRD5THV3aogy1AN6VUZ+AoMAGY5BJVqVZAkXOO8BTgB2enuEbSARZCCCGEEGft9FUg6nWdWtuUUncDawAzsERr/btSaqpz+UKgJ/CWUsoAdgKT3a1XOsBCCCGEEKJeGA1wJzit9WpgdaXXFlb4fxPQrS7rlA6wEEIIIYQ4axp1JnOAfaJpbKUQQgghhBD1REaAhRBCCCFEvbDX/0lwDUI6wEIIIYQQ4qyd4WXQfEI6wEIIIYQQ4qxpVIOcBNcQpAMshBBCCCHqRX1fBq2hSAdYCCGEEEKcNa1piBthNAjpAAshhBBCiHqgsCNTIIQQQgghRDOhkRFgIYQQQgjRzMhVIIQQQgghRLOhUdjlKhBCCCGEEKI5kRFgIYQQQgjRbGjkTnBCCCGEEKJZURhyFQghhBBCCNFcyAiwEEIIIYRodmQEWAghhBBCNBtaKxkBbigR4Wbdqb2frzfjnLL31yBfb8I5qZDjOVrrtr7eDktQsPZrFe61eN3bZnotFoCfD844tqO9Gq/QbvZqPIDMg629HrPwZFqjqDP+KkAHEuzrzWgwJR293+YntPRuu+CP9+uML/z0a0mjqDMVyY0wGkin9n5sXtPB15txThkV09fXm3BO+lp/dNjX2wDg1yqcTlPu81q8NXc+77VYAO3MIV6NB1CsS7wa79vill6NB/DKzTd4PebXG+c0ijoTSDAD1TBfb0aD2ftwf6/H/HzUS16NF2sO9Wo8XzFH72sUdaYpanIdYCGEEEII0fhowC5zgIUQQgghRPOhZAqEEEIIIYRoPhyXQZMRYCGEEEII0YzIrZCFEEIIIUSzoVEyAiyEEEIIIZoXu4wACyGEEEKI5kJrMGQE2Hu+/PYkMx7OxjBg8qQwZk13vej/8TyDyTMyOXC4lMAAxaKXIundIwCAvHyD22dm8fvuEpSCRS9FMiipRaOK54uYSaP6cde82zCZTXyx+BtWPPepy/KQVsHMXHwXMV0jsZ4q5cXJCzj0+xEA3j7wKsWFp7AbdgybwbQBf3dbPl/E9EUZG4sh8R2ZPWYoJmXio59/Y9H6LS7L/zz4Qsb26QGAxWSiS9twBj+3kPziErd5q/PtulM8/EgBhgGTJgYx/W7Xa/fm5dmZMTOPw4cNAgIUL73Ykh49HDe86X9RFiHBCrMZzBbFmtURHpXR23Vm7bpiHnjoOIYdbpkYzMzprtfuPZ5n5y/35XLwsI3AAMWCueH06uHvjGdn2v257NxdilLw2tw2DEwKqDXeT98XsuiJNAwDRt7QmuumtnNZfiLf4JVZqaSnWPEPUPz12Tg6JgQC8PKsVLZ+W0DLNhbmf9nd/ZvplDSwC3fdMxKTSfHFZ9tZ8c4ml+V9zu/A489cT0Z6PgAbvt/NO0s3AHDNnwYwelw/tNYcOpDN80//h1Kr4XFsX3LXVgy6MolbH5+AtmsMm8GCGUv5feNu4rrHMGf5jLJ0UV3aseyRFXzy8upGF/PS2M48ctEwzEqxfO+vvPbr/1yW39l7AOO79gQcbUJ8yzac/9588q2n3OatyXfrSnjU2S5MmNiCadW0C3+bmV/WLrzwYhgJPcpvhGUYmrFjcomMMrN0mfsbtzSHY3dj0OynQCil2gNvAVGAHXhDa/1ypTQKeBkYAxQBt2qtf65LHMPQTJ+dzZoVscRFWxg4OoVxI4NJTCg/eDzzyjH69Q7g4zdj2L3PyvTZWaz9MA6Aex/KZtRlQXy4KBqrVVNUbG9U8XwR02QyMX3+ZGaNfIKc1GPM3/wMm1ZtJWVXalmaibOvYf8vB3ns2udpnxDD9PlTeGDE42XL77/8UQpyC92WzVcxfVFGd7xVZ0xK8dDYy5m87GMyCwr54M5JrNu9n/3Zx8rSLNn4E0s2/gTA0IQu3DLofPKLSzzKW5lhaGbPKWDFe+FER5sZ/cccRo4MIKF7+YHslX+doHcvP95cHM6+ZBuzH8znwxVtypZ/9GEb2oR7/rOaL9qF+2YfZ9XydsRGm/nDmAzGjAqiZ4UyvvBKPn16+bN8SVv27CvlvgeP8fkHkQA88PBxRgxtwbv/r60zXu13mjMMzeuPpvH4ss60ibIw8+r9DBgWRodugWVpPlyQRefEQGYv7Ejq/lMsfCSNJ9/pAsCwa1sz9qY2vHT/EY/fU5NJMf2+K5g14z1ysgqYv+jPbNqwj5RDOS7pdvxyhIdmfeDyWpuIUK66rj9T/u91rFYbcx6/msuG9eKrL34tS/P7vo8B+iqlftNa9wZQSoUDK4BOwCHgT1rr485l/wAmAwbwV631msrbXFt+z8vtvq3Y9s1vbFp1PwCdz+vAnBX3MTnxXlL3pjH1gr+Vref91NfZ+MnmRhfTpBRPDBrOjWs+IONkIauuvJmvU5LZl5dblub13zbz+m+O9Qxr35UpvZLIt57yKG91DEMzZ04B777XmuhoM+P+mMuIkYF0717eLXn1XydI7OXH/1vcmuRkG3MeLGD5ivIO5JLFRcTHWyg84f7OjM3h2N0YOOYAN40pEA25lTZgpta6J3ARME0plVgpzWigm/NxB/BaXYNs3naKrp386NLRD39/xQ3jQ1m15qRLmp17rVw+xHHrxx7d/Dl0xEZmto2CQoP1PxYzeVIYAP7+ilYta799orfj+SJmwoB40pIzyDiYha3UxncrNnLx+CSXNB17xrHtm98AOLInjchObWnV7szvVuXtmL4oowe8Umf6xEWRciyP1OP5lBp2Vu/Yw+U9utaY/o/nJbB6x54zyguwbXspnTqZ6djRgr+/Yvz4Fqz5yvVOanv32RgyxHFQ6BZv4UiqQXb2mY8OervObN1mpUsnC52dZbxufBCfrylySbN7XylDnWVM6OZHyhGDzGyDgkI7G388xS2TgivEq71p3vdLEdEd/Ynq4I+fv4lLxrbkf18XuKQ5klxC34sdI2pxXQPJOlrK8ZxSAHoPCCakVd1uFZvQM4a01GNkpOVhs9n57uudXDzE89Fjs9lEQIAFk1kREOBHbo7rl8eYducD7KuU7e/AN1rrbsA3zuc468UEoBdwBbBAKVVdgarNXxeetBWnTp4q+z8wONDxO3Al5w/rTfr+DLJScqos83XMfhHRHCrI40hhPqV2O/85sIsRHeJrTD++S09WHth1RnlP216pXRg3PpCvvjrlkmbfPoPBQxy/ksTHW0it0C6kpxl8800JEyZ5NiLaHI7djYWBqtPDVxqsA6y1Tj89MqW1LgR2AbGVko0H3tIOPwKtlFLRdYlzNMNG+9jyb4yx0RaOZthc0vRNDOCT1ScAxw55OLWU1DQbBw7baNvGzJ/vzeTCESncPjOTk0W1f8PydjxfxIyIDSc7tfzbe07qMSJi27ikOfDrIYZcMxCAhP7xRHZsS9s4Rxqt4dk1c3h1y3OMuX242/L5IqYvyuiOt+pMu9AQMvLLOx+ZBSeIDKv+dsKBfhaGxHfiq5376pz3tIx0g9jo8oY7OspERrpr5zYx0cLqLxwHv23brKSmGqSlO/ZTpWDCpFxGjs7m7XdcO5U18XadScswiIspL2NstIW0SmU8L9GfVV8UA7B1WwkpqTbS0g0OHbYR0cbM1BnHuHhEOtNm5rqNl5tpIyK6fHQ5IsqP3MxSlzSdegayaY1jKsLeX4rIOmolN931PaiLiLahZGeVf/Y52QVEtK16u9nE3rEsXDqFp16YQMfOjukquTmFfLT8R97993RWfHoPJ0+W8NOWgy75WrfsDI4vgRWNB5Y5/18GXFXh9eVa6xKt9UEgGRhQzWbXlN9jnrQVAIOvGsDinfN48rN/8MLkqt9Lh04YzLrlGxtlzKjgENJPln+26ScLiQqq/lbCgWYLl8Z15otDe+uct6KMdDsxLu2Cmcx01/2+Z6KFL53twvZtVo6mGqQ70zz6aAGzHwzF5GH/qTkcuxuD09cBrsvDV7wyTq2U6gScD1SeGBQLVPwNLpWqB/xaVfOlF1Xp/Zw1vTXH8w0uGH6Y+YvzOL93ABaLwmbT/LyjhKm3tOKntR0IbmHiuX/V/uuYt+P5ImbldTu2wXUjlj/7KaGtgln48/NcdfdokrcdxLA5DvgzhszhrqRZPDjmKa68axTnXdLTbRm9HdMXZayLhqwznpT9tMsSurDtSBr5xSV1zlu23INtmD4thPx8O8NHZrP4zSJ69/bD4jxurPqkDWu/bMt7b4ezdNlJNv1YUnWFVbbJfUxvtwv33R1GXp6dQcPTWbikkL69/bGYwWZotu+wMuXmEP67NpqgIMWL8wuqrtBdvEojKdfd2ZYT+XbuGbuPz97KpUtiC8xnMenNk88+eU8GN143n6m3LmLlR1t47OnrAQgJDWTQkO7c9KdXmXDVKwQG+jFsZG9PwkZqrdOdsdKB0xOdPa0HNeX3mKf7/MZPNzM58V4evfqf3Pr4DS7LLH4WBo1L4vsPN1XJ1zhiVg2oq625MLxDPFszj5JvPT1a63le1zTVbEWlVd01LZj8fM0VI3N4880ievW2YLHA11+fIiLCRJ8+ftWspYZ4zeDY3Tg4pkDU5eErDX4SnFIqBPg3cK/WunKrXl3Xv8ouo5S6A8fPvXSIdd3kuGgLR46Wf6M6mm4jJtI1TViomSXzohwr15quAw7RuYOFomLtmJdzgWPe3LVjQ3hufs1zGX0Rzxcxs1OPlY10AkTEhZOb5pqnqLCYFyYvKHv+9oFXyTiYBUBuuqOS5mUXsPHTzSQMiGfH+l2NKqYvyuip+q4zlpauJ4dkFpwgqmX5CE1kWAhZha4/y502pncCn/+6+4zynhYdbeZohdHQ9Aw7kVGuP+WFhpqYN7eVozBaM2BQNh3aO9JEOdNGRJgZfUUg27eXMuii2k8Q83adiY02k5pWXsaj6TaiK5UxLNTEwnmnf0HQ9BqYRscOFoqLNbHRZvpf4CjTVWODmOumAxwRZSEnvXzENyejlPBK5QsKNXPPP+PK4t1+6R4i4/xrXW9tsrMKaduu/LOPaBtGbs4JlzRFRday/zf/uJ/pM02EtWxBvws6kpGeR36eYwR/ww97SDwvjm+++u1MN8ejelCnFVaoM4EElb3uSVtR0Y71u4juGkVYm9CycwT6j+5H8s8HycvK92hbvB0z42Qh0cHln210cCiZRSeqTTuuSw9WHShv6+qSt6LoaJPLryTpGQbtolw7Q6GhJl6c65h2prVm8KBs2rc3s2rlKdZ+VcK6b7MoKYHCQjv3TM/j5X+1qjFeczh2NxZ2H05rqIsG7XorpfxwHMjf1Vp/XE2SVKB9hedxQFrlRFrrN7TWSVrrpLZtXA8q/fsFknzQysGUUqxWzYqVhYwbFeySJi/fwGp1tI2L3i3gkotaEBZqJqqdhfYxFvYkOxrtbzcUkdi99gOEt+P5IuaeLcnEdosmqlM7LH4Wht4wmE2rtrqkCW4ZhMXPUZFHTxnGjh92UVRYTGBQAC1CHBU2MCiAC0f05dBv7k+08XZMX5TREw1RZyxBrvvKjqMZdAxvTWyrMPzMJsacl8C63QeqBAoJ8CepUxzf7t5f57wV9evrx8GDBikpNqxWzcqVxYwa4dqBzc+3l+2/775XzEUD/QkNNVFUZOfECcfPfkVFdr7/oYSEBPff271dZy7s58/+g6Uccpbxo5VFjBnpOjcxr0IZl753ksEXBRAWaiKynZnYGAt7kx0d2u/Wn6JHt9pHtrr1CSLtUAkZR6yUWu2s/yyfgcPCXNKcKDAotTreu69WHKdX/2CCQs98DuGe3WnEtg8nKrolFouJocMT2bRxr0ua1uHl73FCzxhMJkVBfjFZmQX07BVLQIDjszv/wk5VTp6rQebpKT7Ov1nO1z2qB7Xkr6JinfGjfP/0pK2I6RpV9n/8+Z3x87e4nCB72YQhrFu+wZPy+iTmLznpdG7ZmvYhLfEzmRjXpSdrU5KrpAv18+eiqPZ8VWGZp3kr61upXfjPylOMqKVdeP+9YgY424W//yOUzVvb8d8f2zH/1ZZcPDig1s4vNI9jd2Nw+jJodXn4SkNeBUIBi4FdWuu5NSRbBdytlFoODATyT/9c5SmLRfHK0+0YPfEohgG3TQijV0IAC5flATD1llbs2mfl1r9mYjZBz+7+LJobWZb/5afacdO0DKylms4d/FgyL7KGSL6J54uYdsPO/OmLeebLBzGZTax5cx2Hd6Yy9s4RAHz2+lo69Ixj1rK7MQw7KTtTeXGKY/5Zq8iWPPqx4wxks8XMuvc3sHXNdrdl9HZMX5TRHW/VGcOuefLzb1l08zWYTIqPf/6d5OxcbkjqA8CKrY4z84f3jOe/+w9TXGpzm7c2Fovi6SfCmHjjMQw7TLihBQkJfix72zFyfMtNwexLtvHXe/IwmaF7NwtzX2gFQHa2nT9PcYy22wy4+qpALr8ssKZQLjG93S68+FQ4V03KwjDgpgnBJCb4s+gtR4dkys2h7NlXyh335GIyQY/ufix4sXyE78UnWzP57lxnPAuvvVR1vmdFZovizkdiePTWg9jtMPy61nToHsgX7zk+i9GT2pCafIqX7k/FZFa0jw/gr8/GleV//p4UfvvfSQqO27ht8C4m3hPJyD+F1xQOALuhmT93Dc/MnYjJZGLN579w+GAOY8dfAMBnK3/mD0N7MvbqCzAMO9YSG0898gkAu3emsX7dbhYsmYxh2Nm/N5PVq7bVGs9pFXAL8Kzz78oKr7+nlJoLxOA4KbS6Sx3UlN9jnrQVl1w7kOE3XYpRalBSbOXJCS+V5Q9o4c+FI/owb+objTamoTUPb/qat0Zdj1kpPti3g315udyY0A+Ad/dsB2BUx+78cPQQxbZSt3ndsVgUTzwRxk03Oi4deIOzXXj7bcevBDfdFERyso0Z9+RjNkO3bhb++cKZn4TcHI7djUVTuQqEcjd/74xXrNQQYD2wA8clnQBmAx0AtNYLnQf8+TjO4i0CbtNab61mdWWS+gbqzWs6NMg2N1ejYvr6ehPOSV/rj37SWie5T+nQUHWmRUx73WnKfWdQgjOz7s7nvRYLoJ259pPwGkKxdj8PuT59W9ygVx+p1is33+A+UT3asWcFmTk7SnFMb8gEHgE+BT7AUQdSgOu11scAlFIPAn/GceLcvVrrL5yvLwIWaq23KqXa1JS/NmEqXA9Uw+q5hI3H3kX9vR5zw6iX3CeqR7Fm9yfinQvM0fvqdJxpaG16ttVjlo6vU553LlrskzI02Aiw1noD1c/TqphGA9MaahuEaEqkzojm7LyEG8jM2fFrNQfCanuiWuungKeqeX1Khf9za8ovhGjezok7wQkhhBBCCN+Tk+CEEEIIIUSz0VDXAVZKXaGU2qOUSlZKVbmhjVKqpVLqP0qpX5RSvyulbnO3ThkBFkIIIYQQ9aK+T4Jz3uXxVWAEjivAbFFKrdJa76yQbBqwU2s9TinVFtijlHpXa22tZpWAdICFEEIIIUR9aJi7uw0AkrXWBwCcV0EaD1TsAGsg1HmieAhwjKp3lnQhHWAhhBBCCHHWNA0yB7i6Oz8OrJRmPo7LHqYBocANWuta7x0tc4CFEEIIIUS9OIM5wBFKqa0VHndUWqUnd34cBWzHcV3wfsB8pVQYtZARYCGEEEIIcdZOnwRXRzlurgPsyZ0fbwOedV4qNFkpdRDoQfU3yAFkBFgIIYQQQtSTBrgKxBagm1Kqs1LKH5iAY7pDRSk4r/mtlIoEEoADta1URoCFEEIIIcRZ09T/SXBaa5tS6m5gDWAGlmitf1dKTXUuXwg8ASxVSu3AMWViltY6p7b1SgdYCCGEEELUi4a4EYbWejWwutJrCyv8nwaMrMs6pQMshBBCNCPdp2zxeswhi2Z4Nd6GUS95NR5ArDnU6zEbHX1Gc4B9QjrAQgghhBDirJ3hSXA+IR1gIYQQQghRL6QDLIQQQgghmo2GOAmuoUgHWAghhBBC1AstHWAhhBBCCNGcNMRVIBqCdICFEEIIIcRZ003oKhByJzghhBBCCNGsnBMjwF9+e5IZD2djGDB5Uhizpoe7LD+eZzB5RiYHDpcSGKBY9FIkvXsEAJCXb3D7zCx+312CUrDopUgGJbVoVPF8ETNpVD/umncbJrOJLxZ/w4rnPnVZHtIqmJmL7yKmayTWU6W8OHkBh34/AsDbB16luPAUdsOOYTOYNuDvbsvni5i+KGNjMSS+I7PHDMWkTHz0828sWu96XdA/D76QsX16AGAxmejSNpzBzy0kv7jEbd7qfLvuFA8/UoBhwKSJQUy/O8RleV6enRkz8zh82CAgQPHSiy3p0cMPgP4XZRESrDCbwWxRrFkd4VEZvV1n1q4r5oGHjmPY4ZaJwcyc3rJSPDt/uS+Xg4dtBAYoFswNp1cPf2c8O9Puz2Xn7lKUgtfmtmFgUkCt8X76vpBFT6RhGDDyhtZcN7Wdy/IT+QavzEolPcWKf4Dir8/G0TEhEICXZ6Wy9dsCWraxMP/L7u7fTKekgV24656RmEyKLz7bzop3Nrks73N+Bx5/5noy0vMB2PD9bt5ZugGAa/40gNHj+qG15tCBbJ5/+j+UWg2PY/uSu7Zi0JVJ3Pr4BLRdY9gMFsxYyu8bdxPXPYY5y8uvfxvVpR3LHlnBJy+vxh1vx/RFGS+N7cwjFw3DrBTL9/7Ka7/+z2X5nb0HML5rT8DRDsW3bMP5780n33rKbd7qfLeuhEed7dCEiS2YVk079LeZ+WXt0AsvhpHgbIcADEMzdkwukVFmli5r7TYe+Ka/4GvNfg6wUmoJMBbI0lr3rmZ5S+AdoINzO17QWr9Z1ziGoZk+O5s1K2KJi7YwcHQK40YGk5hQfvB45pVj9OsdwMdvxrB7n5Xps7NY+2EcAPc+lM2oy4L4cFE0VqumqNjeqOL5IqbJZGL6/MnMGvkEOanHmL/5GTat2krKrtSyNBNnX8P+Xw7y2LXP0z4hhunzp/DAiMfLlt9/+aMU5Ba6LZuvYvqijO54q86YlOKhsZczednHZBYU8sGdk1i3ez/7s4+VpVmy8SeWbPwJgKEJXbhl0PnkF5d4lLcyw9DMnlPAivfCiY42M/qPOYwcGUBC9/IDyyv/OkHvXn68uTicfck2Zj+Yz4cr2pQt/+jDNrQJ9/wHK1+0C/fNPs6q5e2IjTbzhzEZjBkVRM8KZXzhlXz69PJn+ZK27NlXyn0PHuPzDyIBeODh44wY2oJ3/19bZzztNt7rj6bx+LLOtImyMPPq/QwYFkaHboFlaT5ckEXnxEBmL+xI6v5TLHwkjSff6QLAsGtbM/amNrx0/xGP31OTSTH9viuYNeM9crIKmL/oz2zasI+UQ653G93xyxEemvWBy2ttIkK56rr+TPm/17Fabcx5/GouG9aLr7741aPY1dUNpVQ4sALoBBwC/qS1Pl5N3iuAl3HcQnWR1vpZjwuNZ23Ftm9+Y9Oq+wHofF4H5qy4j8mJ95K6N42pF/ytbD3vp77Oxk82N7qYPimjUjwxaDg3rvmAjJOFrLryZr5OSWZfXm5Zmtd/28zrvznWNax9V6b0SiLfesqjvJUZhmbOnALefa810dFmxv0xlxEjA+nevbwb9Oq/TpDYy4//t7g1yck25jxYwPIV5R3WJYuLiI+3UHii9vpZMaa3+wu+13SuAtGQUyCWAlfUsnwasFNr3RcYCryolPKva5DN207RtZMfXTr64e+vuGF8KKvWnHRJs3OvlcuHBAHQo5s/h47YyMy2UVBosP7HYiZPCgPA31/RqqW5UcXzRcyEAfGkJWeQcTALW6mN71Zs5OLxSS5pOvaMY9s3vwFwZE8akZ3a0qpdy+pW5xFvx/RFGT2wFC/UmT5xUaQcyyP1eD6lhp3VO/ZweY+uNab/43kJrN6x54zyAmzbXkqnTmY6drTg768YP74Fa74qcUmzd5+NIUMcB4Vu8RaOpBpkZ5/56KC368zWbVa6dLLQ2VnG68YH8fmaIpc0u/eVMtRZxoRufqQcMcjMNigotLPxx1PcMim4Qrzam+Z9vxQR3dGfqA7++PmbuGRsS/73dYFLmiPJJfS92DHCFdc1kKyjpRzPKQWg94BgQlq5b3sqSugZQ1rqMTLS8rDZ7Hz39U4uHuL56LHZbCIgwILJrAgI8CM3p05fHpdStW78HfhGa90N+Mb53IVSygy8CowGEoGJSqnEugT2pK04dfJU2f+BwYGOiZCVnD+sN+n7M8hKyamyzNcxfVHGfhHRHCrI40hhPqV2O/85sIsRHeJrTD++S09WHth1RnkBtldqh8aND+Srr065pNm3z2DwEEeTGh9vIbVCO5SeZvDNNyVMmOT5CKwv+guNgdaqTg9fabAOsNb6B6DmYSHHDUNClVIKCHGmtdU1ztEMG+1jy7/BxUZbOJrhupq+iQF8svoE4NghD6eWkppm48BhG23bmPnzvZlcOCKF22dmcrKo9m9Y3o7ni5gRseFkp5Z/k85JPUZEbBuXNAd+PcSQawYCkNA/nsiObWkb50ijNTy7Zg6vbnmOMbcPd1s+X8T0RRnd8VadaRcaQkZ+eecjs+AEkWEh1aYN9LMwJL4TX+3cV+e8p2WkG8RGlzfc0VEmMtJdO7eJiRZWf+E4GG3bZiU11SAt3bGfKgUTJuUycnQ2b7/j2qmsibfrTFqGQVxMeRljoy2kVSrjeYn+rPqiGICt20pISbWRlm5w6LCNiDZmps44xsUj0pk2M9dtvNxMGxHR5aPLEVF+5GaWuqTp1DOQTWscUxH2/lJE1lEruel13l3KY7QNJTur/LPPyS4gom3VW78m9o5l4dIpPPXCBDp2dkxXyc0p5KPlP/Luv6ez4tN7OHmyhJ+2HPQ4dg11YzywzPn/MuCqarIOAJK11ge01lZguTOfxzxpKwAGXzWAxTvn8eRn/+CFya9VWT50wmDWLd/YKGP6ooxRwSGknyzfn9JPFhIVVP2thAPNFi6N68wXh/bWOe9pGel2YlzaITOZ6a71rGeihS+d7dD2bVaOphqkO9M8+mgBsx8MxVSH/pov+gu+dvpOcHV5+IovT4KbD/QE0oAdwD1a6zp/utV8CUVVej9nTW/N8XyDC4YfZv7iPM7vHYDForDZND/vKGHqLa34aW0HgluYeO5fVX5B82k8X8SsvG7HNrhuxPJnPyW0VTALf36eq+4eTfK2gxg2xwF/xpA53JU0iwfHPMWVd43ivEt6ui2jt2P6ooz1oF7qjCdlP+2yhC5sO5JGfnFJnfOWLfdgG6ZPCyE/387wkdksfrOI3r39sDiPG6s+acPaL9vy3tvhLF12kk0/llRdYZVtch/T2+3CfXeHkZdnZ9DwdBYuKaRvb38sZrAZmu07rEy5OYT/ro0mKEjx4vyCqit0F6/SpYeuu7MtJ/Lt3DN2H5+9lUuXxBaYz2LSmyefffKeDG68bj5Tb13Eyo+28NjT1wMQEhrIoCHduelPrzLhqlcIDPRj2Mgqs3zqKlJrne7cjnSgXTVpYoGK8zxSna95zNN9fuOnm5mceC+PXv1Pbn38BpdlFj8Lg8Yl8f2Hm6rkawwxfVFGqrlUlq62tYDhHeLZmnmUfOvpEVvP85Yvr2YLKq3mrmnB5OdrrhiZw5tvFtGrtwWLBb7++hQRESb69PGrZi21xPRBf8HntKPcdXn4ii9PghsFbAcuB7oCa5VS67XWVVp+pdQdwB0AHWJdNzku2sKRo+XfqI6m24iJdE0TFmpmybwowFGpuw44ROcOFoqKtWNezgWOeXPXjg3hufm1DcB5P54vYmanHisb6QSIiAsnN801T1FhMS9MXlD2/O0Dr5JxMAuA3HRHJc3LLmDjp5tJGBDPjvW7GlVMX5SxHpxRnbG0dD1ZI7PgBFEty0dLIsNCyCp0/VnutDG9E/j8191nlPe06GgzRyuMhqZn2ImMcv0pLzTUxLy5rQDH/jtgUDYd2jvSRDnTRkSYGX1FINu3lzLootpPEPN2nYmNNpOaVl7Go+k2oiuVMSzUxMJ5p39B0PQamEbHDhaKizWx0Wb6X+Ao01Vjg5jrpgMcEWUhJ718xDcno5TwSuULCjVzzz/jyuLdfukeIuPqPGOmTHZWIW3blX/2EW3DyM054ZKmqMha9v/mH/czfaaJsJYt6HdBRzLS88jPc4zgb/hhD4nnxfHNV7+d8fZ4qLrhpWoPuRXrTCBBZa970lZUtGP9LqK7RhHWJrTsHIH+o/uR/PNB8rLyPdpob8f0RRkzThYSHVy+P0UHh5JZdKLatOO69GDVgfL2tS55y9JEm1x+lUnPMGgX5ToGGBpq4sW5jmluWmsGD8qmfXszq1aeYu1XJaz7NouSEigstHPP9Dxe/lerWmP6or/QGDSV6wD7cgT4NuBj7ZAMHAR6VJdQa/2G1jpJa53Uto3rQaV/v0CSD1o5mFKK1apZsbKQcaOCXdLk5RtYrY42b9G7BVxyUQvCQs1EtbPQPsbCnmRHo/3thiISu9d+gPB2PF/E3LMlmdhu0UR1aofFz8LQGwazadVWlzTBLYOw+Dkq8ugpw9jxwy6KCosJDAqgRYijwgYGBXDhiL4c+s39iTbejumLMtaDM6ozliDXfWXH0Qw6hrcmtlUYfmYTY85LYN3uA1XWERLgT1KnOL7dvb/OeSvq19ePgwcNUlJsWK2alSuLGTXCtQObn28v23/ffa+Yiwb6ExpqoqjIzokTjkHuoiI73/9QQkKC++/t3q4zF/bzZ//BUg45y/jRyiLGjHSdK5hXoYxL3zvJ4IsCCAs1EdnOTGyMhb3Jjg7td+tP0aNb7SNN3foEkXaohIwjVkqtdtZ/ls/AYWEuaU4UGJRaHe/dVyuO06t/MEGhZz6HcM/uNGLbhxMV3RKLxcTQ4Yls2rjXJU3r8PL3OKFnDCaToiC/mKzMAnr2iiUgwPHZnX9hpyonz52BTKVUNIDzb1Y1aVKB9hWex+H4BaWKinXGj/L905O2IqZrVNn/8ed3xs/f4nKC7GUThrBu+QaPC+btmL4o4y856XRu2Zr2IS3xM5kY16Una1OSq6QL9fPnoqj2fFVhmad5K+pbqR36z8pTjKilHXr/vWIGONuhv/8jlM1b2/HfH9sx/9WWXDw4wG3nF3zTX/A1TdOZA+zLEeAUYBiwXikVCSQAtR9Jq2GxKF55uh2jJx7FMOC2CWH0Sghg4bI8AKbe0opd+6zc+tdMzCbo2d2fRXMjy/K//FQ7bpqWgbVU07mDH0vmRdYQyTfxfBHTbtiZP30xz3z5ICaziTVvruPwzlTG3jkCgM9eX0uHnnHMWnY3hmEnZWcqL05xzAdrFdmSRz92nBFstphZ9/4Gtq7Z7raM3o7pizLWg3qpM4Zd8+Tn37Lo5mswmRQf//w7ydm53JDUB4AVWx1n5g/vGc9/9x+muNTmNm9tLBbF00+EMfHGYxh2mHBDCxIS/Fj2tmPk+JabgtmXbOOv9+RhMkP3bhbmvtAKgOxsO3+e4hhttxlw9VWBXH5ZYE2hXGJ6u1148alwrpqUhWHATROCSUzwZ9Fbjg7ClJtD2bOvlDvuycVkgh7d/VjwYvmI24tPtmby3bnOeBZee6nq/MuKzBbFnY/E8OitB7HbYfh1renQPZAv3nN8FqMntSE1+RQv3Z+KyaxoHx/AX5+NK8v//D0p/Pa/kxQct3Hb4F1MvCeSkX8KrykcAHZDM3/uGp6ZOxGTycSaz3/h8MEcxo6/AIDPVv7MH4b2ZOzVF2AYdqwlNp565BMAdu9MY/263SxYMhnDsLN/byarV22rNZ4HVgG3AM86/66sJs0WoJtSqjNwFJgATKpLEE/aikuuHcjwmy7FKDUoKbby5ISXyvIHtPDnwhF9mDf1jUYb0xdlNLTm4U1f89ao6zErxQf7drAvL5cbE/oB8O6e7QCM6tidH44eothW6jZvbSwWxRNPhHHTjY5LFd7gbIfeftvxq8RNNwWRnGxjxj35mM3QrZuFf75wdic9+6K/4HtN5yoQyt38vTNesVLv4zhTPQLIBB4B/AC01guVUjE4zuyNxvEz1bNa63fcrTepb6DevKZDg2xzczUqpq+vN+Gc9LX+6CetdZL7lA4NVWdaxLTXnabcV+ftP1Pr7nzea7EA2plrPwmvIRRr9/OQ69O3xQ169ZFqvXLzDe4T1bOvN86pts7UUDc+BT7AcVnAFOB6rfUxZz1ZpLUe48w7BpiH4zJoS7TWT7nbjjAVrgeqYfVRJOG0d1F/r8bbMOol94nqWay59hPxGoI5el+djjMNLahbjO720uQ65fl13JM+KUODjQBrrSe6WZ4GjGyo+EI0NVJnhKheLXWjSi/VWU/GVHi+GnB/VwYhRL1o9jfCEEIIIYQQzYfjyg7SARZCCCGEEM1IU5kDLB1gIYQQQghRL3x5bd+6kA6wEEIIIYSoFzIFQgghhBBCNBsa317bty6kAyyEEEIIIepFE5kB4dM7wQkhhBBCCOF1MgIshBBCCCHOnlwGTQghhBBCNDtNZA6EdICFEEIIIUS9kBFgIYQQQgig+5QtXo03ZNEMr8YD2DDqJa/HbIzkOsBCCCGEEKLZ0MgIsBBCCCGEaE40IB1gIYQQQgjRnMgUCCGEEEII0bxIB1gIIYQQQjQfcitkIYQQQgjR3MgIsBBCCCGEaDaa0J3gTL7eACGEEEIIcY7QdXx4QCl1hVJqj1IqWSn192qW/00ptd35+E0pZSilwmtb5zkxAvzltyeZ8XA2hgGTJ4Uxa7prmY/nGUyekcmBw6UEBigWvRRJ7x4BAOTlG9w+M4vfd5egFCx6KZJBSS0aVTxfxEwa1Y+75t2GyWzii8XfsOK5T12Wh7QKZubiu4jpGon1VCkvTl7Aod+PAPD2gVcpLjyF3bBj2AymDaiyrzaKmL4oY2MxJL4js8cMxaRMfPTzbyxa73qR+j8PvpCxfXoAYDGZ6NI2nMHPLSS/uMRt3up8u+4UDz9SgGHApIlBTL87xGV5Xp6dGTPzOHzYICBA8dKLLenRww+A/hdlERKsMJvBbFGsWR3hURm9XWfWrivmgYeOY9jhlonBzJzeslI8O3+5L5eDh20EBigWzA2nVw9/Zzw70+7PZefuUpSC1+a2YWBSQK3xfvq+kEVPpGEYMPKG1lw3tZ3L8hP5Bq/MSiU9xYp/gOKvz8bRMSEQgJdnpbL12wJatrEw/8vu7t9Mp6SBXbjrnpGYTIovPtvOinc2uSzvc34HHn/mejLS8wHY8P1u3lm6AYBr/jSA0eP6obXm0IFsnn/6P5RaDY9j+5K7tmLQlUnc+vgEtF1j2AwWzFjK7xt3E9c9hjnLy2/IENWlHcseWcEnL69udDGbQxkvje3MIxcNw6wUy/f+ymu//s9l+Z29BzC+a0/A0e7Ft2zD+e/NJ996ym3emny3roRHnW3fhIktmFZN2/e3mfllbd8LL4aR4Gz7AAxDM3ZMLpFRZpYua+1RTN+r3xFgpZQZeBUYAaQCW5RSq7TWO0+n0Vo/DzzvTD8OmKG1PlbbehusA6yUWgKMBbK01r1rSDMUmAf4ATla60vrGscwNNNnZ7NmRSxx0RYGjk5h3MhgEhPKDx7PvHKMfr0D+PjNGHbvszJ9dhZrP4wD4N6Hshl1WRAfLorGatUUFdsbVTxfxDSZTEyfP5lZI58gJ/UY8zc/w6ZVW0nZlVqWZuLsa9j/y0Eeu/Z52ifEMH3+FB4Y8XjZ8vsvf5SC3EK3ZfNVTF+U0R1v1RmTUjw09nImL/uYzIJCPrhzEut272d/dnlbsWTjTyzZ+BMAQxO6cMug88kvLvEob2WGoZk9p4AV74UTHW1m9B9zGDkygITu5Y38K/86Qe9efry5OJx9yTZmP5jPhyvalC3/6MM2tAn3/AcrX7QL980+zqrl7YiNNvOHMRmMGRVEzwplfOGVfPr08mf5krbs2VfKfQ8e4/MPIgF44OHjjBjagnf/X1tnvNqHRQxD8/qjaTy+rDNtoizMvHo/A4aF0aFbYFmaDxdk0TkxkNkLO5K6/xQLH0njyXe6ADDs2taMvakNL91/xOP31GRSTL/vCmbNeI+crALmL/ozmzbsI+VQjku6Hb8c4aFZH7i81iYilKuu68+U/3sdq9XGnMev5rJhvfjqi189jl8dd3VGKaWAl4ExQBFwq9b657rE8KSt2PbNb2xadT8Anc/rwJwV9zE58V5S96Yx9YK/la3n/dTX2fjJ5kYXs1mUUSmeGDScG9d8QMbJQlZdeTNfpySzLy+3LM3rv23m9d8c6xnWvitTeiWRbz3lUd7qGIZmzpwC3n2vNdHRZsb9MZcRIwPp3r286/Xqv06Q2MuP/7e4NcnJNuY8WMDyFeVf1pcsLiI+3kLhiSYysRYaYg7wACBZa30AQCm1HBgP7Kwh/UTgfXcrbcgpEEuBK2paqJRqBSwArtRa9wKuP5Mgm7edomsnP7p09MPfX3HD+FBWrTnpkmbnXiuXDwkCoEc3fw4dsZGZbaOg0GD9j8VMnhQGgL+/olVLc6OK54uYCQPiSUvOIONgFrZSG9+t2MjF45Nc0nTsGce2b34D4MieNCI7taVVu5bVrc4j3o7pizJ6YCleqDN94qJIOZZH6vF8Sg07q3fs4fIeXWtM/8fzEli9Y88Z5QXYtr2UTp3MdOxowd9fMX58C9Z8VeKSZu8+G0OGODqn3eItHEk1yM4+89FBb9eZrdusdOlkobOzjNeND+LzNUUuaXbvK2Wos4wJ3fxIOWKQmW1QUGhn44+nuGVScIV4tTfN+34pIrqjP1Ed/PHzN3HJ2Jb87+sClzRHkkvoe7FjtCmuayBZR0s5nlMKQO8BwYS0ct/2VJTQM4a01GNkpOVhs9n57uudXDzE89Fjs9lEQIAFk1kREOBHbk69fHlcSi11BhgNdHM+7gBeq2sAT9qKUydPlf0fGBxY7YVQzx/Wm/T9GWSl5FRZ5uuYzaGM/SKiOVSQx5HCfErtdv5zYBcjOsTXmH58l56sPLDrjPKetr1S2zdufCBffXXKJc2+fQaDhzh+CYqPt5Baoe1LTzP45psSJkxy/ytxo1L/UyBigYrf1lOdr1WhlArC0Sb8291KG6wDrLX+Aaht+HkS8LHWOsWZPutM4hzNsNE+tvzbVGy0haMZNpc0fRMD+GT1CcBxYDycWkpqmo0Dh220bWPmz/dmcuGIFG6fmcnJotpHerwdzxcxI2LDyU4t/2abk3qMiNg2LmkO/HqIIdcMBCChfzyRHdvSNs6RRmt4ds0cXt3yHGNuH+62fL6I6YsyuuOtOtMuNISM/PLOR2bBCSLDQqpNG+hnYUh8J77aua/OeU/LSDeIjS7vbEVHmchId+3cJiZaWP2F48CwbZuV1FSDtHTHfqoUTJiUy8jR2bz9jmunsiberjNpGQZxMeVljI22kFapjOcl+rPqi2IAtm4rISXVRlq6waHDNiLamJk64xgXj0hn2sxct/FyM21ERJePLkdE+ZGbWeqSplPPQDatcUxF2PtLEVlHreSmu74HdRHRNpTsrPLPPie7gIi2oVXSJfaOZeHSKTz1wgQ6dnZMV8nNKeSj5T/y7r+ns+LTezh5soSfthw84205zYM6Mx54Szv8CLRSSkXXJYYnbQXA4KsGsHjnPJ787B+8MLlqP3vohMGsW76xUcZsDmWMCg4h/WT5/pt+spCooKr7L0Cg2cKlcZ354tDeOuetKCPdToxL22cmM921bvdMtPCls+3bvs3K0VSDdGeaRx8tYPaDoZiaxjllDqfvBFeXB0QopbZWeNxRaa3VvQM1dZ3HARvdTX8A354E1x1orZT6Tin1k1Lq5jNZSXV3HFGV3qpZ01tzPN/gguGHmb84j/N7B2CxKGw2zc87Sph6Syt+WtuB4BYmnvvX8UYVzxcxK6/bsQ2uG7H82U8JbRXMwp+f56q7R5O87SCGzXHAnzFkDnclzeLBMU9x5V2jOO+Snm7L6O2YvihjPaiXOuNJ2U+7LKEL246kkV9cUue8Zcs92Ibp00LIz7czfGQ2i98sondvPyzO/uuqT9qw9su2vPd2OEuXnWTTjyVVV1hlm9zH9Ha7cN/dYeTl2Rk0PJ2FSwrp29sfixlshmb7DitTbg7hv2ujCQpSvDi/oOoK3cWrdIy47s62nMi3c8/YfXz2Vi5dEltgPotJb5589sl7MrjxuvlMvXURKz/awmNPO36kCAkNZNCQ7tz0p1eZcNUrBAb6MWxktbN86pvHI0c18XSf3/jpZiYn3sujV/+TWx+/wWWZxc/CoHFJfP/hpir5GkPM5lDG6vpQuoY+1PAO8WzNPEq+9fRored5XdNUsxWVVnXXtGDy8zVXjMzhzTeL6NXbgsUCX399iogIE336+FWzlnNOjtY6qcLjjUrLU4H2FZ7HAWk1rGsCHkx/AN+eBGcBLgSGAS2ATUqpH7XWeysndH4buAOgQ6zrJsdFWzhytHxU42i6jZhI1zRhoWaWzIsCHBWs64BDdO5goahYO+YHXuCYN3ft2BCem1/7lwZvx/NFzOzUY2UjnQARceHkprnmKSos5oXJC8qev33gVTIOOgYkc9MdnYW87AI2frqZhAHx7Fi/q1HF9EUZ68EZ1RlLS9cTJzILThDVsnz0IjIshKxC1+kBp43pncDnv+4+o7ynRUebOVphNDQ9w05klOvP76GhJubNbQU49t8Bg7Lp0N6RJsqZNiLCzOgrAtm+vZRBF9V+gpi360xstJnUtPIyHk23EV2pjGGhJhbOO/0LgqbXwDQ6drBQXKyJjTbT/wJHma4aG8RcNx3giCgLOenlI745GaWEVypfUKiZe/4ZVxbv9kv3EBnnX+t6a5OdVUjbduWffUTbMHJzTrikKSqylv2/+cf9TJ9pIqxlC/pd0JGM9Dzy8xwj+Bt+2EPieXF889VvZ7w9HvJ45KhinQkkqOx1T9qKinas30V01yjC2oSWnSPQf3Q/kn8+SF5Wvkcb7e2YzaGMGScLiQ4u33+jg0PJLDpRbdpxXXqw6kB5e16XvBVFR5tcfglKzzBoF+U67hgaauLFuY6pdVprBg/Kpn17M6tWnmLtVyWs+zaLkhIoLLRzz/Q8Xv5XK7dxfa0BboW8BeimlOoMHMXRyZ1UOZFSqiVwKfB/nqzUlyPAqcCXWuuTWusc4Aegb3UJtdZvnP5m0LaN60Glf79Akg9aOZhSitWqWbGykHGjgl3S5OUbWK2OT2TRuwVcclELwkLNRLWz0D7Gwp5kR6P97YYiErvXfoDwdjxfxNyzJZnYbtFEdWqHxc/C0BsGs2nVVpc0wS2DsPg5Drijpwxjxw+7KCosJjAogBYhjo5DYFAAF47oy6Hf3J9o4+2YvihjPTijOmMJct1XdhzNoGN4a2JbheFnNjHmvATW7T5QZR0hAf4kdYrj293765y3on59/Th40CAlxYbVqlm5sphRI1w7sPn59rL99933irlooD+hoSaKiuycOOH4ObCoyM73P5SQkOD+e7u368yF/fzZf7CUQ84yfrSyiDEjXeft5VUo49L3TjL4ogDCQk1EtjMTG2Nhb7KjQ/vd+lP06Fb7qE+3PkGkHSoh44iVUqud9Z/lM3BYmEuaEwUGpVbHe/fViuP06h9MUGjd5v1WtGd3GrHtw4mKbonFYmLo8EQ2bXT97tU6vPw9TugZg8mkKMgvJiuzgJ69YgkIcHx251/YqcrJcw3E45GjinXGj/L905O2IqZrVNn/8ed3xs/f4nKC7GUThrBu+QaPN9rbMZtDGX/JSadzy9a0D2mJn8nEuC49WZuSXCVdqJ8/F0W156sKyzzNW1nfSm3ff1aeYkQtbd/77xUzwNn2/f0foWze2o7//tiO+a+25OLBAU2i8wvU+xxgrbUNuBtYA+wCPtBa/66UmqqUmloh6dXAV1rr2kdlnHw5ArwSmK+UsgD+wEDgpbquxGJRvPJ0O0ZPPIphwG0TwuiVEMDCZXkATL2lFbv2Wbn1r5mYTdCzuz+L5kaW5X/5qXbcNC0Da6mmcwc/lsyLrCGSb+L5IqbdsDN/+mKe+fJBTGYTa95cx+GdqYy9cwQAn72+lg4945i17G4Mw07KzlRenOKYm9UqsiWPfuw4O9dsMbPu/Q1sXbPdbRm9HdMXZawH9VJnDLvmyc+/ZdHN12AyKT7++XeSs3O5IakPACu2Os7MH94znv/uP0xxqc1t3tpYLIqnnwhj4o3HMOww4YYWJCT4sextRxt1y03B7Eu28dd78jCZoXs3C3NfaAVAdradP09xjLbbDLj6qkAuvyywplAuMb3dLrz4VDhXTcrCMOCmCcEkJviz6C3HwXrKzaHs2VfKHffkYjJBj+5+LHixfPTrxSdbM/nuXGc8C6+9VHUuZEVmi+LOR2J49NaD2O0w/LrWdOgeyBfvOT6L0ZPakJp8ipfuT8VkVrSPD+Cvz8aV5X/+nhR++99JCo7buG3wLibeE8nIP9V6yUzshmb+3DU8M3ciJpOJNZ//wuGDOYwdfwEAn638mT8M7cnYqy/AMOxYS2w89cgnAOzemcb6dbtZsGQyhmFn/95MVq/aVmu8erIKuNt51vhAIF9rnV6XFXjSVlxy7UCG33QpRqlBSbGVJyeUV8uAFv5cOKIP86ZW/lW38cRsDmU0tObhTV/z1qjrMSvFB/t2sC8vlxsT+gHw7p7tAIzq2J0fjh6i2FbqNq87FoviiSfCuOlGx+URb3C2fW+/7fgl5KabgkhOtjHjnnzMZujWzcI/X2jQE629owFuhKG1Xg2srvTawkrPl+I4MdYjyt38vTOllHofGApEAJnAIzgu3VS20UqpvwG3AXZgkdZ6nrv1JvUN1JvXdGiQbW6uRsVUO4goztLX+qOftNZJ7lM6NFSdaRHTXneacl9dN/+Mrbvzea/FAmhnrv0kvIZQrN3PQ65P3xZ7/6D4ys03uE9Uz77eOKde64zzMmjzcZwVXgTcprXeWv3ayoWpcD1QDat7AUSjsXdRf6/H3DCqzuMRZ61DXEad6kxDC+gUp6MfvKdOeQ7f8YBPytBgI8Ba64kepCm7cLEQzZ3UGSHqxl2d0Y4Rnmle2hwhRB3u7uZr58Sd4IQQQgghhK+pBpkC0RCkAyyEEEIIIeqHjAALIYQQQohmRTrAQgghhBCiWZEOsBBCCCGEaDZO3wq5CZAOsBBCCCGEqBdKRoCFEEIIIUSz0kQ6wLXeClkpFa+UGlzN65copbo23GYJ0TQV6RPk6aq3eJU6I5qDouJc8goOV3n9eP4hiord3zlLCCG8pdYOMDAPKKzm9WLnMiFEBXvZjrn6H1akzohz3t6DqzGbA6q8bjb5sffg6mpyCCHONUrX7eEr7qZAdNJa/1r5Ra31VqVUp4bZJCG3Jm66iikiVLWq8npzqjOFXm7Q2nk3nE+EmU75ehM8UlxynNDgqCqvh4XGUlyS5/0NEs1W9ylbvB5zyKIZXo8Js3wQ041z5CS4wFqWtajPDRHiXGDHqG2x1BlxTrPbbbUsK/XilgghfKIJ3QrZ3RSILUqp2yu/qJSaDPzUMJskRNMVRjhH9YEqr0udEc1BWEgsRzOqjrwdzdxKWHCMD7ZICCGq524E+F7gE6XUjZQfvJMAf+DqBtwuIZqk7vTlV/5Luk4BiFNKvYjUGdFMdO/8R37d/S7p2b8QFhILQMGJo2ht0KfHJB9vnRDCK5rICHCtHWCtdSZwsVLqMqC38+XPtdbfNviWCdEEBahA+nM5x3QWP/ODFTiE1BnRTAT4h9C/z50cyzvAiaJMACJadye8lVwARYjm4py6DrDWeh2wroG3RYhzRrhqB5osrfW/fL0tQnhbeKsuhLfq4uvNEEL4wrnUARZCCCGEEMIt6QALIYQQQojmwtfX9q0L6QALIYQQQoj6cY5cB7hJ+PLbk8x4OBvDgMmTwpg1Pdxl+fE8g8kzMjlwuJTAAMWilyLp3cNxt6K8fIPbZ2bx++4SlIJFL0UyKKn2y7V6Ox5A0qh+3DXvNkxmE18s/oYVz33qsjykVTAzF99FTNdIrKdKeXHyAg79fgSAtw+8SnHhKeyGHcNmMG3A3xtdvOZSxsZiSHxHZo8ZikmZ+Ojn31i03vXSVX8efCFj+/QAwGIy0aVtOIOfW0h+cYnbvNX5Yd0pnnq0AMOA6ycGcee0EJfl+Xl2/nF/HkcOG/gHKJ55oSXde/iVLTcMzTV/zCEyyswbS8Mrr75a3q6na9cV88BDxzHscMvEYGZOb1kpnp2/3JfLwcM2AgMUC+aG06uHvzOenWn357JzdylKwWtz2zAwqeod1Sr63/cnmf9YNoYd/nhDGDf+xbV8hfkGzz2QSdrhUvwDFA/8M5IuCQFkpZXy9MxMjmXbMJkUYyeGcd1trT16T5MGduGue0ZiMim++Gw7K97Z5LK8z/kdePyZ68lIzwdgw/e7eWfpBgCu+dMARo/rh9aaQweyef7p/1BqrfW62Y2Gu7Zi0JVJ3Pr4BLRdY9gMFsxYyu8bdxPXPYY5y8tvjhDVpR3LHlnBJy+7vyuet2NKGRumjJfGduaRi4ZhVorle3/ltV//57L8zt4DGN+1J+Boa+NbtuH89+aTbz3lNm+j1dxHgJVSS4CxQJbWunct6foDPwI3aK0/qmscw9BMn53NmhWxxEVbGDg6hXEjg0lMKD94PPPKMfr1DuDjN2PYvc/K9NlZrP0wDoB7H8pm1GVBfLgoGqtVU1Rsb1TxAEwmE9PnT2bWyCfIST3G/M3PsGnVVlJ2pZalmTj7Gvb/cpDHrn2e9gkxTJ8/hQdGPF62/P7LH6Ugt7q7Wvs+XnMpozveqjMmpXho7OVMXvYxmQWFfHDnJNbt3s/+7GNlaZZs/IklGx1XPhya0IVbBp1PfnGJR3krMwzNY3MKePO9cKKizVw7NodhIwKI717ewV04/wQ9e/mxYFE4+5NtPDYnn7eWtylbvmzxSbrGWzhxwrOW1Rftwn2zj7NqeTtio838YUwGY0YF0bNCGV94JZ8+vfxZvqQte/aVct+Dx/j8g0gAHnj4OCOGtuDd/9fWGa/2chqG5uWHs3nh7VjaRlmYOj6FwcOD6dStvHzvvHqM+MQAnnw9hsP7rbz8cBZz343DbFHc9WAE3XsHUnTCzh3jUkgaEuSStzomk2L6fVcwa8Z75GQVMH/Rn9m0YR8ph3Jc0u345QgPzfrA5bU2EaFcdV1/pvzf61itNuY8fjWXDevFV19UudFonSilrgBeBszAIq31s5WW9wDeBC4AHtRav1DXGJ60Fdu++Y1Nq+4HoPN5HZiz4j4mJ95L6t40pl7wt7L1vJ/6Ohs/2dzoYkoZG6iMSvHEoOHcuOYDMk4WsurKm/k6JZl9ebllaV7/bTOv/+ZY17D2XZnSK4l86ymP8jZWTWUKhLsbYZyNpcAVtSVQSpmB54A1Zxpk87ZTdO3kR5eOfvj7K24YH8qqNSdd0uzca+XyIUEA9Ojmz6EjNjKzbRQUGqz/sZjJk8IA8PdXtGppblTxABIGxJOWnEHGwSxspTa+W7GRi8cnuaTp2DOObd/8BsCRPWlEdmpLq3Ytq1tdo4vni5i+KKMHluKFOtMnLoqUY3mkHs+n1LCzesceLu9R82Wq/nheAqt37DmjvAC/bi+lYyczHTpa8PdX/PHKFnz9VYlLmuR9NgYNdnTAusZbOHrEICfbMTqYkW7w3bclXD8xyOMyeruebt1mpUsnC52dZbxufBCfrylySbN7XylDhzjKmNDNj5QjBpnZBgWFdjb+eIpbJgVXiFd707z7l1PEdvQjpoMffv6Ky8eFsnGta/kOJ1u54GJH+Tp29Scj1caxbBtt2lno3ttxk8+gEBMd4/3Jyaj5Dm6nJfSMIS31GBlpedhsdr77eicXD+nuNt9pZrOJgAALJrMiIMCP3Jyz+/LorAuvAqOBRGCiUiqxUrJjwF+BOnd8T/OkrTh1svxW1YHBgaCr9gDOH9ab9P0ZZKXkVFnm65hSxoYpY7+IaA4V5HGkMJ9Su53/HNjFiA7xNaYf36UnKw/sOqO8jYqu48NHGqwDrLX+AUfjU5vpwL+BrDONczTDRvvY8oHs2GgLRys15n0TA/hk9QnAcWA8nFpKapqNA4dttG1j5s/3ZnLhiBRun5nJyaLaR3q8HQ8gIjac7NTyb305qceIiG3jkubAr4cYcs1AABL6xxPZsS1t4xxptIZn18zh1S3PMeb24Y0uXnMpozveqjPtQkPIyC/vfGQWnCAyLKTatIF+FobEd+KrnfvqnLcsTYZBVEx5BzIq2kRmhutP3z16WvjqC8fB55dtVtKOGmSkO+rGU48W8MDsMEx1aK28XU/TMgziKpQxNtpCWrprGc9L9GfVF8UAbN1WQkqqjbR0g0OHbUS0MTN1xjEuHpHOtJm5buNlZ9hoG11evrZRFrIrla9rzwDWr3GUb9f2U2QcLa2SJj21lH07S+jZr7a73jtEtA0lO6v8s8/JLiCibWiVdIm9Y1m4dApPvTCBjp0jAMjNKeSj5T/y7r+ns+LTezh5soSfthx0G9ONAUCy1vqA1toKLAfGV0ygtc7SWm8Bzvg+zJ60FQCDrxrA4p3zePKzf/DC5NeqLB86YTDrlm9slDGljPUfDyAqOIT0k+V1Jv1kIVFBVesMQKDZwqVxnfni0N46521UdPmJcJ4+fKUhR4BrpZSKxXFnrIVns55qvqChKs2/njW9NcfzDS4Yfpj5i/M4v3cAFovCZtP8vKOEqbe04qe1HQhuYeK5fx1vVPGqW79jO1w3ZPmznxLaKpiFPz/PVXePJnnbQQyb4wA8Y8gc7kqaxYNjnuLKu0Zx3iU9G1W85lLGs1VfdcaTsp92WUIXth1JI7+4pM55y5e734Y7p4VQkG/nylHZvL20iJ69/DBbYN3Xp2jTxkTvPn5VV3KWMb3dLtx3dxh5eXYGDU9n4ZJC+vb2x2IGm6HZvsPKlJtD+O/aaIKCFC/OL3BTQPfxJk1tTWG+weQxh/l4WR7degVgNpcnKjpp55G/pHP3Q20JDnX/S5Qnn33yngxuvG4+U29dxMqPtvDY09cDEBIayKAh3bnpT68y4apXCAz0Y9jIGmf5eCoWOFLhearztXrl6T6/8dPNTE68l0ev/ie3Pn6DyzKLn4VB45L4/sNNVfI1hphSxvqP54xaNWYNQ57DO8SzNfMo+dbTo9Ce5210msgIsC9PgpsHzNJaG6q6PbMCpdQdwB0AHWJdNzku2sKRo+WjGkfTbcREuqYJCzWzZF4U4Njhuw44ROcOFoqKtWN+4AWO0Y9rx4bw3PzaB+C8HQ8gO/VY2cgjQERcOLlprvmKCot5YfKCsudvH3iVjIOOQcLcdMfBOy+7gI2fbiZhQDw71u9qNPGaSxnrwTzOoM5YWrqe4JRZcIKoluUjCZFhIWQVuv58ftqY3gl8/uvuM8p7WlS0mYy08tHQjHQ77SJdO1whoSaendsKcNSZyy/Opn17M5+vKuabtaf4fl0JJSWaE4V27v/rcV54pfaTtrxdT2OjzaRWKOPRdBvRUa5lDAs1sXDe6V8QNL0GptGxg4XiYk1stJn+FzimR1w1Noi5bjrAbaMtZKeXly87w0ZEpfIFh5r5+/Pl5ZtwySGi2zvS2Eo1j/wlneHjQ/nDFbWP4JfFyCqkbbvyzz6ibRi5OSdc0hQVWcv+3/zjfqbPNBHWsgX9LuhIRnoe+XmOaSEbfthD4nlxfPPVbx7FrkF1leCMD6cV60wg5dNtPGkrKtqxfhfRXaMIaxNado5A/9H9SP75IHlZ+R5ti7djShnrPx5AxslCooPL60x0cCiZRSeqTTuuSw9WHSg/htQlb6PTRPrpPhsBBpKA5UqpQ8B1wAKl1FXVJdRav6G1TtJaJ7Vt43pQ6d8vkOSDVg6mlGK1alasLGTcqGCXNHn5Blar4xNZ9G4Bl1zUgrBQM1HtLLSPsbAn2dFof7uhiMTu/rVutLfjAezZkkxst2iiOrXD4mdh6A2D2bRqq0ua4JZBWPwcB7fRU4ax44ddFBUWExgUQIsQx4E8MCiAC0f05dBvR6rE8GW85lLGenBGdcYS5Lp/7jiaQcfw1sS2CsPPbGLMeQms232gyjpCAvxJ6hTHt7v31zlvRef19ePQIYMjKTasVs3nq4oZNsL1hKuCfHtZnfng/WKSBvoTEmri/r+HsX5LJOs2teOlV1tx0eAAt51f8H49vbCfP/sPlnLIWcaPVhYxZqTrVSPyKpRx6XsnGXxRAGGhJiLbmYmNsbA32fEr/XfrT9GjW+0j3gl9Akk9ZCX9SCmlVs23/ynk4uGu5SssMCh1xvt8eQF9B7QgONSM1pp/zsqkQ7w/f5ri2dUfAPbsTiO2fThR0S2xWEwMHZ7Ipo17XdK0Di/fhoSeMZhMioL8YrIyC+jZK5aAAEd9Ov/CTlVOnjsDqUD7Cs/jgLQzXVnFOuNH+f7pSVsR0zWq7P/48zvj529xOUH2sglDWLd8g8fb4u2YUsaGKeMvOel0btma9iEt8TOZGNelJ2tTkqukC/Xz56Ko9nxVYZmneRujpjIFwmcjwFrrzqf/V0otBT7TWn9a1/VYLIpXnm7H6IlHMQy4bUIYvRICWLgsD4Cpt7Ri1z4rt/41E7MJenb3Z9HcyLL8Lz/VjpumZWAt1XTu4MeSeZE1RPJNPAC7YWf+9MU88+WDmMwm1ry5jsM7Uxl75wgAPnt9LR16xjFr2d0Yhp2Unam8OMUxd6lVZEse/dhx9qrZYmbd+xvYumZ7o4rXXMp4tuqrzhh2zZOff8uim6/BZFJ8/PPvJGfnckNSHwBWbHWcmT+8Zzz/3X+Y4lKb27y1sVgUDz8RxuT/O4ZhwHU3tKBbgh/vv+0YOZ54UzD7k208cG8eJjPEd7Pw9POt6lqsKjG93S68+FQ4V03KwjDgpgnBJCb4s+gtx8Fzys2h7NlXyh335GIyQY/ufix4sXw06sUnWzP57lxnPAuvvVR1bmLlePc81o6/3XwUux1GXx9G5+4BrHzXUb7xN7YiJdnK0zMzMZmgUzd/HnjOUYYdW0/x1SeFdEnwZ/KYwwDc/rcILrosuKZwANgNzfy5a3hm7kRMJhNrPv+FwwdzGDv+AgA+W/kzfxjak7FXX4Bh2LGW2HjqkU8A2L0zjfXrdrNgyWQMw87+vZmsXrWt1nge2AJ0U0p1Bo4CE4BJZ7vSyjxpKy65diDDb7oUo9SgpNjKkxNeKssf0MKfC0f0Yd7UNxptTCljw5TR0JqHN33NW6Oux6wUH+zbwb68XG5M6AfAu3u2AzCqY3d+OHqIYlup27yi/ih38/fOeMVKvQ8MBSKATOARwA9Aa72wUtqlOA7mbi/plNQ3UG9e06G+N7dRGRXT19ebIOrB1/qjn7TWSe5TOjRUnWkR0153mnKf5xt+llbd8bzXYgF0tXj2E359KtYl7hPVo60ltV+irCE8+X+3eD3m1xvn1LXOjMExNcgMLNFaP6WUmgqOOqOUigK2AmGAHTgBJGqta51jEqbC9UA17AxLIZqrvYv6ez1myuRZdaozDS0wtr3ueGfdjjd7H7nPJ2VosBFgrfXEOqS9taG2Q4imQuqMEHWjtV4NrK702sIK/2fgmBohhPCGJnQrZF/OARZCCCGEEMLrzolbIQshhBBCiEagiYwASwdYCCGEEELUD+kACyGEEEKI5kLRdOYASwdYCCGEEELUD+kACyGEEEKIZqMJXQVCOsBCCCGEEKJ+NJEOsFwGTQghhBBC1A9dx4cHlFJXKKX2KKWSlVJ/ryHNUKXUdqXU70qp792tU0aAhRBCCCFEvajvKRBKKTPwKjACSAW2KKVWaa13VkjTClgAXKG1TlFKtXO3XhkBFkIIIYQQ9aP+R4AHAMla6wNaayuwHBhfKc0k4GOtdQqA1jrL3UplBNiNUTF9fb0Joh6sSfvF6zHN0V4PKYQQwke6T9ni9ZgpXo/oRh2mNdRBLHCkwvNUYGClNN0BP6XUd0Ao8LLW+q3aViodYCGEEEIIUS/OYApEhFJqa4Xnb2it36i4ymryVI5iAS4EhgEtgE1KqR+11ntrCiodYCGEEEIIUT/q3gHO0Von1bI8FWhf4XkckFZNmhyt9UngpFLqB6AvUGMHWOYACyGEEEKIeqF03R4e2AJ0U0p1Vkr5AxOAVZXSrAQuUUpZlFJBOKZI7KptpTICLIQQQggh6kc9zwHWWtuUUncDawAzsERr/btSaqpz+UKt9S6l1JfAr4AdWKS1/q229UoHWAghhBBCnL2GOQkOrfVqYHWl1xZWev488Lyn65QOsBBCCCGEOGuK6s9Ya4ykAyyEEEIIIeqH3ApZCCGEEEKIxuecGAH+8tuTzHg4G8OAyZPCmDU93GX58TyDyTMyOXC4lMAAxaKXIundIwCAvHyD22dm8fvuEpSCRS9FMiipRa3xkkb14655t2Eym/hi8TeseO5Tl+UhrYKZufguYrpGYj1VyouTF3Dod8c1nN8+8CrFhaewG3YMm8G0AdXe0trnMZtDGb293zQmQ+I7MnvMUEzKxEc//8ai9a4XcP/z4AsZ26cHABaTiS5twxn83ELyi0vc5q3OD+tO8dSjBRgGXD8xiDunhbgsz8+z84/78zhy2MA/QPHMCy3p3sOvbLlhaK75Yw6RUWbeWBpeefXV8vbnu3ZdMQ88dBzDDrdMDGbm9JaV4tn5y325HDxsIzBAsWBuOL16+Dvj2Zl2fy47d5eiFLw2tw0DkwJqjfe/708y/7FsDDv88YYwbvyLa/kK8w2eeyCTtMOl+AcoHvhnJF0SAshKK+XpmZkcy7ZhMinGTgzjuttae/SeJg3swl33jMRkUnzx2XZWvLPJZXmf8zvw+DPXk5GeD8CG73fzztINAFzzpwGMHtcPrTWHDmTz/NP/odRqeBTX19y1TYOuTOLWxyeg7RrDZrBgxlJ+37ibuO4xzFk+oyxdVJd2LHtkBZ+8vBp3vB1TynhulLExqO9bITeUBusAK6WWAGOBLK1172qW3wjMcj49AfxFa13n23UZhmb67GzWrIglLtrCwNEpjBsZTGJC+cHjmVeO0a93AB+/GcPufVamz85i7YdxANz7UDajLgviw0XRWK2aomJ7rfFMJhPT509m1sgnyEk9xvzNz7Bp1VZSdqWWpZk4+xr2/3KQx659nvYJMUyfP4UHRjxetvz+yx+lILfQ4zJ6O2ZzKKO39xtPeKvOmJTiobGXM3nZx2QWFPLBnZNYt3s/+7OPlaVZsvEnlmz8CYChCV24ZdD55BeXeJS3MsPQPDangDffCycq2sy1Y3MYNiKA+O7lHdyF80/Qs5cfCxaFsz/ZxmNz8nlreZuy5csWn6RrvIUTJzxrWb39+RqG5r7Zx1m1vB2x0Wb+MCaDMaOC6FmhjC+8kk+fXv4sX9KWPftKue/BY3z+QSQADzx8nBFDW/Du/2vrjFd7OQ1D8/LD2bzwdixtoyxMHZ/C4OHBdOpWXr53Xj1GfGIAT74ew+H9Vl5+OIu578ZhtijuejCC7r0DKTph545xKSQNCXLJWx2TSTH9viuYNeM9crIKmL/oz2zasI+UQzku6Xb8coSHZn3g8lqbiFCuuq4/U/7vdaxWG3Mev5rLhvXiqy9+rTXmaUqpK4CXcZwBvkhr/Wyl5cq5fAxQBNyqtf7ZuazWeuWOJ23Ttm9+Y9Oq+wHofF4H5qy4j8mJ95K6N42pF/ytbD3vp77Oxk82N7qYUsZzo4yNRhPpADfkFIilwBW1LD8IXKq17gM8AbxRS9oabd52iq6d/OjS0Q9/f8UN40NZteakS5qde61cPiQIgB7d/Dl0xEZmto2CQoP1PxYzeVIYAP7+ilYtzbXGSxgQT1pyBhkHs7CV2vhuxUYuHu96/eaOPePY9o3j6htH9qQR2aktrdq1rG51HvF2zOZQRm/vNx5aihfqTJ+4KFKO5ZF6PJ9Sw87qHXu4vEfXGtP/8bwEVu/Yc0Z5AX7dXkrHTmY6dLTg76/445Ut+PqrEpc0yftsDBrs6IB1jbdw9IhBTrZjdDAj3eC7b0u4fmKQx2X09ue7dZuVLp0sdHaW8brxQXy+psglze59pQwd4ihjQjc/Uo4YZGYbFBTa2fjjKW6ZFFwhXu1N8+5fThHb0Y+YDn74+SsuHxfKxrWu5TucbOWCix3l69jVn4xUG8eybbRpZ6F770AAgkJMdIz3JyfDVms8gISeMaSlHiMjLQ+bzc53X+/k4iHd3eY7zWw2ERBgwWRWBAT4kZvj2ZdVpZQZeBUYDSQCE5VSiZWSjQa6OR93AK9VWLaU2utVrTxpm06dPFX2f2BwIOiqPYDzh/UmfX8GWSk5VZb5OqaU8dwoY6Oh6/jwkQbrAGutfwBqHBbSWv9Xa33c+fRHHHf2qLOjGTbax5YPZMdGWzhaqTHvmxjAJ6tPAI4D4+HUUlLTbBw4bKNtGzN/vjeTC0ekcPvMTE4W1T7SExEbTnZqbtnznNRjRMS2cUlz4NdDDLnGcZvqhP7xRHZsS9u4Ns5yw7Nr5vDqlucYc/twj8ro7ZjNoYze3m884a060y40hIz88s5HZsEJIsNCqk0b6GdhSHwnvtq5r855y9JkGETFlHcgo6JNZGa4/vTdo6eFr75wHAh+2WYl7ahBRrrjPX3q0QIemB2GqQ6tlbc/37QMg7gKZYyNtpCW7lrG8xL9WfVFMQBbt5WQkmojLd3g0GEbEW3MTJ1xjItHpDNtZq7beNkZNtpGl5evbZSF7Erl69ozgPVrHOXbtf0UGUdLq6RJTy1l384SevYLrDUeQETbULKzyj/7nOwCItqGVkmX2DuWhUun8NQLE+jYOQKA3JxCPlr+I+/+ezorPr2HkydL+GnLQbcxnQYAyVrrA1prK7AcGF8pzXjgLe3wI9BKKRUN7uuVO560TQCDrxrA4p3zePKzf/DC5NeqLB86YTDrlm9slDGljPUfz1cxfa6ON8Hw5XSJxnIS3GTgizPJWM2XJVSla3DMmt6a4/kGFww/zPzFeZzfOwCLRWGzaX7eUcLUW1rx09oOBLcw8dy/jlddYS3rdmyD60Ysf/ZTQlsFs/Dn57nq7tEkbzuIYXMcDGcMmcNdSbN4cMxTXHnXKM67pKfbMno7ZnMoo7f3mwZwxnXGk/f6tMsSurDtSBr5xSV1zlu+3P023DkthIJ8O1eOyubtpUX07OWH2QLrvj5FmzYmevfxq7qSs4xZn5+vJ/HuuzuMvDw7g4ans3BJIX17+2Mxg83QbN9hZcrNIfx3bTRBQYoX5xe4KaD7eJOmtqYw32DymMN8vCyPbr0CMJvLExWdtPPIX9K5+6G2BIe6/wXDk88+eU8GN143n6m3LmLlR1t47OnrAQgJDWTQkO7c9KdXmXDVKwQG+jFspMezEWKBIxWepzpfq2uaM+LpPr/x081MTryXR6/+J7c+foPLMoufhUHjkvj+w01V8jWGmFLG+o/nq5iNQhMZAfb5SXBKqctwHMyH1JLmDhw/a9Eh1nWT46ItHDlaPqpxNN1GTKRrmrBQM0vmRQGOna/rgEN07mChqFg75gde4Bj9uHZsCM/Nr32gIDv1WNmoI0BEXDi5aa55igqLeWHygrLnbx94lYyDWQDkpjsOpHnZBWz8dDMJA+LZsb7Wu/V5PWZzKKO395v6VNc6Y2npeoJTZsEJolqWj9xFhoWQVej68/lpY3on8Pmvu88o72lR0WYy0spHQzPS7bSLdO1whYSaeHZuK8DxXl9+cTbt25v5fFUx36w9xffrSigp0ZwotHP/X4/zwiu1n7Tl7c83NtpMaoUyHk23ER3lWsawUBML553+xULTa2AaHTtYKC7WxEab6X+BY3rEVWODmOumA9w22kJ2enn5sjNsRFQqX3Comb8/X16+CZccIrq9I42tVPPIX9IZPj6UP1xR+wh+WYysQtq2K//sI9qGkZtzwiVNUZG17P/NP+5n+kwTYS1b0O+CjmSk55Gf55gWsuGHPSSeF8c3X9V6o6bTqrusaOXDpidpag9Soc4EUj7dxpO2qaId63cR3TWKsDahZeck9B/dj+SfD5KXle/Rtng7ppSx/uP5KmZj0FROgvPpCLBSqg+wCBivtc6tKZ3W+g2tdZLWOqltG9eDSv9+gSQftHIwpRSrVbNiZSHjRgW7pMnLN7BaHZ/IoncLuOSiFoSFmolqZ6F9jIU9yY5G+9sNRSR29691m/dsSSa2WzRRndph8bMw9IbBbFq11SVNcMsgLH6OA83oKcPY8cMuigqLCQwKoEWI46AaGBTAhSP6cui3I1Vi+Dpmcyijt/eb+nImdcYS5FquHUcz6BjemthWYfiZTYw5L4F1uw9UWUdIgD9JneL4dvf+Ouet6Ly+fhw6ZHAkxYbVqvl8VTHDRriecFWQby97rz94v5ikgf6EhJq4/+9hrN8SybpN7Xjp1VZcNDjAbecXvP/5XtjPn/0HSznkLONHK4sYM9L1qhF5Fcq49L2TDL4ogLBQE5HtzMTGWNibXArAd+tP0aNb7SPeCX0CST1kJf1IKaVWzbf/KeTi4a7lKywwKHXG+3x5AX0HtCA41IzWmn/OyqRDvD9/muLZ1R8A9uxOI7Z9OFHRLbFYTAwdnsimjXtd0rQOL9+GhJ4xmEyKgvxisjIL6NkrloAAR/09/8JOVU6eq0Uq0L7C8zgg7QzS1KpinfGjfP/0pG2K6RpV9n/8+Z3x87e4nJB72YQhrFu+weNt8XZMKeO5UcZGQ0aAa6eU6gB8DNyktd7rLn1NLBbFK0+3Y/TEoxgG3DYhjF4JASxclgfA1FtasWuflVv/monZBD27+7NobmRZ/pefasdN0zKwlmo6d/BjybzIGiI52A0786cv5pkvH8RkNrHmzXUc3pnK2DtHAPDZ62vp0DOOWcvuxjDspOxM5cUpjjk9rSJb8ujHjrM6zRYz697fwNY1292W0dsxm0MZvb3f1If6qjOGXfPk59+y6OZrMJkUH//8O8nZudyQ1AeAFVsdZ+YP7xnPf/cfprjU5jZvbSwWxcNPhDH5/45hGHDdDS3oluDH+287Ro4n3hTM/mQbD9ybh8kM8d0sPP18qzMtXllMb36+FovixafCuWpSFoYBN00IJjHBn0VvOQ5kU24OZc++Uu64JxeTCXp092PBi+UjQy8+2ZrJd+c641l47aWq8wQrx7vnsXb87eaj2O0w+vowOncPYOW7jvKNv7EVKclWnp6ZickEnbr588BzjjLs2HqKrz4ppEuCP5PHHAbg9r9FcNFlwTWFA8BuaObPXcMzcydiMplY8/kvHD6Yw9jxFwDw2cqf+cPQnoy9+gIMw461xMZTj3wCwO6daaxft5sFSyZjGHb2781k9apttcarYAvQTSnVGTgKTAAmVUqzCrhbKbUcGAjka63TPQ1QG0/apkuuHcjwmy7FKDUoKbby5ISXyvIHtPDnwhF9mDfV83NWvR1TynhulLGxaCojwMrd/L0zXrFS7wNDgQggE3gE8APH/ZuVUouAa4HDziw2rXVSNatykdQ3UG9e06FBtrk6o2L6ei2WaDhr0up8tbCzZo7e95Mn+/RpDVVnWsS0152m3FfHrT9zq+7w+Fbs9aKrxbOf8OtTsS5xn6gebS2p/RJlDeHJ/7vF6zG/3jin2jqjlBoDzMNxGbQlWuunlFJToaxuKGA+jqs9FAG3aa23OvNWqVda68W1bUeYCtcD1bB6K5cQDeVr/VGdjjMNLahte93j2rodb7a9fp9PytBgI8Ba64lulk8BpjRUfCGaGqkzQlRPa70aWF3ptYUV/tfAtBry1lqvhBD1rImMAPv8JDghhBBCCNH0KZrOFAjpAAshhBBCiPohHWAhhBBCCNGcqAY6t6y+SQdYCCGEEEKcPR9f2qwupAMshBBCCCHqhcwBFkIIIYQQzUsT6QD79E5wQgghhBBCeJuMAAshhBBCiHohUyCEEEIIIUTzIh3ghrH31yC5PbEQwqcK7aVejRdq8sER5b/bvR9TCNG0aRkBFkIIIYQQzY10gIUQQgghRHMht0IWQgghhBDNj9wJTgghhBBCNCcyAiyEEEIIIZoPuRWyEEIIIYRobpTd11vgGekACyGEEEKI+iEjwEIIIYQQojmROcBelDSqH3fNuw2T2cQXi79hxXOfuiwPaRXMzMV3EdM1EuupUl6cvIBDvx8B4O0Dr1JceAq7YcewGUwb8PdGF0/K2DAxv/z2JDMezsYwYPKkMGZND3dZfjzPYPKMTA4cLiUwQLHopUh69wgAIC/f4PaZWfy+uwSlYNFLkQxKauFRORuDIfEdmT1mKCZl4qOff2PR+i0uy/88+ELG9ukBgMVkokvbcAY/t5D84hK3eavzw7pTPPVoAYYB108M4s5pIS7L8/Ps/OP+PI4cNvAPUDzzQku69/ArW24Ymmv+mENklJk3loZXXn21vP35frvuFA8/4ijjpIlBTL/btYx5eXZmzMzj8GGDgADFSy+2pIezjP0vyiIkWGE2g9miWLM6wm35/vtdMS88lodhwFUTgrntrjCX5QX5dh772zFSD9sICFA8/Hxr4hP8KTmluf1PWVitGsOmGTYmiKn3tXQbD9zX0T6XJvL4p7PIOJgFwIZP/sc7T3zkUd7GzN22D7oyiVsfn4C2awybwYIZS/l9427iuscwZ/mMsnRRXdqx7JEVfPLy6kYXU8p4bpTR5zRyFQil1BJgLJClte5dzXIFvAyMAYqAW7XWP9c1jslkYvr8ycwa+QQ5qceYv/kZNq3aSsqu1LI0E2dfw/5fDvLYtc/TPiGG6fOn8MCIx8uW33/5oxTkFjbKeFLGholpGJrps7NZsyKWuGgLA0enMG5kMIkJAWVpnnnlGP16B/DxmzHs3mdl+uws1n4YB8C9D2Uz6rIgPlwUjdWqKSo++0lPXqszSvHQ2MuZvOxjMgsK+eDOSazbvZ/92cfK0izZ+BNLNv4EwNCELtwy6Hzyi0s8yluZYWgem1PAm++FExVt5tqxOQwbEUB89/IO7sL5J+jZy48Fi8LZn2zjsTn5vLW8TdnyZYtP0jXewokTnjWs3v58DUMze04BK94LJzrazOg/5jByZAAJFcr4yr9O0LuXH28uDmdfso3ZD+bz4YryMn70YRvahJs8Lt+zDx1nwbvtiIwyc9OVmVw6vAVdKsRbMr+AhEQ/XnwjgoPJpTz30HEWvt8O/wBY+H5bgoJNlJZqJl+XxeChgZx3QUAtET2rowA71u/ioSufPaO8QJhSag9gBhZprV1WVFsdUEpd4VzmklcpdT3wKNATGKC13ur2Da7jtm/75jc2rbofgM7ndWDOivuYnHgvqXvTmHrB38rW837q62z8ZHOjiyllPDfK2Fg0lRFgz1rbM7MUuKKW5aOBbs7HHcBrZxIkYUA8ackZZBzMwlZq47sVG7l4fJJLmo4949j2zW8AHNmTRmSntrRq59mIh6/j+SJmcyjj5m2n6NrJjy4d/fD3V9wwPpRVa066pNm518rlQ4IA6NHNn0NHbGRm2ygoNFj/YzGTJzlG3Pz9Fa1ams9oOypZihfqTJ+4KFKO5ZF6PJ9Sw87qHXu4vEfXGtP/8bwEVu/Yc0Z5AX7dXkrHTmY6dLTg76/445Ut+PqrEpc0yftsDBrs6IB1jbdw9IhBTrYBQEa6wXfflnD9xCCPy+jtz3fb9lI6dTLT0VnG8eNbsKZSGffuszFkiKOM3eItHEk1yHaWsa5+326lfSc/4jpY8PNXjBwXxHdri13SHNhXSv/BgQB0jvcjLdVGbraBUoqgYEfTb7NpbKXacfV6Nzypo2eTVztGjTrg2M8TgYlKqcRKq6q2DiilzMCrNeT9DbgG+MGjjT2DbT918lTZ/4HBgdWOgJ0/rDfp+zPISslpdDGljOdGGRsNXceHjzRYB1hr/QNQ87AQjAfe0g4/Aq2UUtF1jRMRG052am7Z85zUY0TEtnFJc+DXQwy5ZiAACf3jiezYlrZxbZzbCc+umcOrW55jzO3DG108KWPDxDyaYaN9bPkPILHRFo5m2FzS9E0M4JPVJwBHh+pwaimpaTYOHLbRto2ZP9+byYUjUrh9ZiYni85+BNhbdaZdaAgZ+eUj5ZkFJ4gMC6k2baCfhSHxnfhq57465y1Lk2EQFVPegYyKNpGZ4drx69HTwldfOA4Ev2yzknbUICPd8Z4+9WgBD8wOw1SH1srbn29GukFsdHkZo6NMZKS7ljEx0cJqZxm3bbOSmmqQ5iyjUjBhUi4jR2fz9jtFbsuXlWEQWSFeZLSZ7ErvafdEP9Z94VjXb9tLyDhqkOVMYxiaiaMzGHFBGhddEsh559c++gue1VGAxEHdWbjteZ76fDYdE+M8zpvv2PVLtNYHtNZWYDmOfb6imurAACC5urxa611a6z1uC3iW5R581QAW75zHk5/9gxcmV/1uOnTCYNYt39goY0oZ6z+er2L62uk7wdXl4SsNOQLsTixwpMLzVOdrdaKqGbnQlb5BLX/2U0JbBbPw5+e56u7RJG87iGFzHAhmDJnDXUmzeHDMU1x51yjOu6Rno4onZWyYmNVNUaq8DbOmt+Z4vsEFww8zf3Ee5/cOwGJR2Gyan3eUMPWWVvy0tgPBLUw896/jbstYD7xWZ067LKEL246kkV9cUue85cvdb8Od00IoyLdz5ahs3l5aRM9efpgtsO7rU7RpY6J3H7+qKznLmPX5+Vb3DlSON31aCPn5doaPzGbxm0X07u2HxdlHX/VJG9Z+2Zb33g5n6bKTbPqxpOoK6xjv1r+EUVBgZ+LoDFYsPUFCLz/Mzj6z2ax4/4sovvgxht+2W0neY601XnXrh6qfffLPB7mx011MPf9vrJz/BY998oDHeUsoBqi4IdXt3zXVgXqpG9XxdJ/f+OlmJifey6NX/5NbH7/BZZnFz8KgcUl8/+GmRhlTylj/8XwV0+e0rvvDR3x5Elx1P7pV+04ope7A8XMXgbj+DJqdeqxsFBAgIi6c3DTXQbSiwmJemLyg7PnbB14tO0kjN91xYMvLLmDjp5tJGBDPjvW7atxob8eTMjZMzLhoC0eOlo8IHk23ERPpWh3CQs0smRcFOBqtrgMO0bmDhaJi7ZhXeoHj5+Vrx4bw3PzaBm7rzRnVGUvL1i7LMgtOENUytOx5ZFgIWYWu0wNOG9M7gc9/3X1GeU+LijaTkVY+OpmRbqddpOuUgpBQE8/ObeUokNZcfnE27dub+XxVMd+sPcX360ooKdGcKLRz/1+P88IrrmWqzNufb3S0maMVRnzTM+xERrmWMTTUxLwKZRwwKJsO7R1popxpIyLMjL4ikO3bSxl0Uc2jspFRZjIrxMtMN4io5j199IXTv5Boxg1JJ6a963sQ2tJE0qAA/vvdKeIT/Gsto6d19LTNX2xj+qtTCGsT6lHeGlTev2uqAx7XjZrUdJyp67bvWL+L6K5RhLUJLTsnof/ofiT/fJC8rHyPtsXbMaWM9R/PVzHPVTXN8a+wfCiwEjjofOljrfXj1MKXI8CpQPsKz+OAtOoSaq3f0Fonaa2T/HA9KOzZkkxst2iiOrXD4mdh6A2D2bTK9RyH4JZBWPwcDf/oKcPY8cMuigqLCQwKoEWI4yAXGBTAhSP6cui3I9TG2/GkjA0Ts3+/QJIPWjmYUorVqlmxspBxo4Jd0uTlG1itjmPooncLuOSiFoSFmolqZ6F9jIU9yY7Bqm83FJHYvfbOQz05ozpjCXIt146jGXQMb01sqzD8zCbGnJfAut0HqqwjJMCfpE5xfLt7f53zVnReXz8OHTI4kmLDatV8vqqYYSNc63FBvr3svf7g/WKSBvoTEmri/r+HsX5LJOs2teOlV1tx0eAAt51f8P7n26+vHwcPGqQ4y7hyZTGjKpUxv0IZ332vmIsG+hMaaqKoyM6JE46pEEVFdr7/oYSEhNrHJhL7+nPkYClHU2yUWjVf/aeIS0e4XqWiMN9OqTPeJ8tPcsGAAEJCTRzPNSjMd8Q7dcrO/zacolO8+xF2T+po68hWZf8n9I/HZDJRkFvoUd4AWgBUfKOr279rqgMe142a1HSc8WTbY7pGlf0ff35n/PwtLifkXjZhCOuWb/B4W7wdU8p4bpSxsajvKRBu5vhXtF5r3c/5qLXzC74dAV4F3K2UWg4MBPK11ul1XYndsDN/+mKe+fJBTGYTa95cx+GdqYy9cwQAn72+lg4945i17G4Mw07KzlRenOKYY9MqsiWPfuw4y9JsMbPu/Q1sXbO9UcWTMjZMTItF8crT7Rg98SiGAbdNCKNXQgALl+UBMPWWVuzaZ+XWv2ZiNkHP7v4smhtZlv/lp9px07QMrKWazh38WDIvsoZI9ape6oxh1zz5+bcsuvkaTCbFxz//TnJ2Ljck9QFgxdZfARjeM57/7j9McanNbd7aWCyKh58IY/L/HcMw4LobWtAtwY/333aMHE+8KZj9yTYeuDcPkxniu1l4+vlWdS1WlZje/HwtFsXTT4Qx8cZjGHaYcEMLEhL8WOYs4y03BbMv2cZf73GUsXs3C3NfcJQxO9vOn6c4fsGwGXD1VYFcflmg23gPPN6au2/OxjA04/8UQtfufnz0jmNO83X/F8LB5FIevu8YJjN0iffj4ecdl4HLyTJ45D7Hdmq7ZvjYIP4wzP0l/Dypo3+47iLGTh2JYTOwFlt5auJLteatKIzWAIFKqc7AUWACMKnSZlRbB5RS2UA3N3nPiCflvuTagQy/6VKMUoOSYitPTnipLH9AC38uHNGHeVPfaLQxpYznRhkbjfqf1VA2xx/AWf/HAzvPZqXK3fy9M16xUu8DQ4EIIBN4BPAD0FovdF7OZj6Os96LgNs8uTxNmArXA9WwBtlmce5ak/aL12Oao/f9pLX27DR5Gq7OtIhprztNua/uBThDq+543muxALpaaj8JryFkGSe8Gi/N8P5YxaxOA70e82v9UTKOw6cZWKK1fkopNRXc1wGl1BhgXsW8ztevBv4FtAXygO1a61G1bYccZ0RT8bX+qE7HmYYW2ipOX3DJPXXK88NnD9RaBqXUdcAVWuspzuc3AQO11ndXSDMU+DeOX4PSgPu11r/XFrfBWlWt9UQ3yzUwraHiC9HUSJ0RgvzKB0Kt9cIK/9dYB7TWq4EqdwnQWn8CfFLP2ymEqI4G7HUeWI1QSlUczHlDa11x2NuTOf4/Ax211iecX4Y/xXG5xBqdE3eCE0IIIYQQjUDdJxbkuBnFdjvHX2tdUOH/1UqpBUqpCK11jRdP9uVJcEIIIYQQ4hzSANcB3oJzjr9Syh/HHP9VLjGVinJOkUIpNQBH/7bWE1RkBFgIIYQQQtSPej63TGttU0rdDayhfI7/7xXPDwCuA/6ilLIBxcAE7eYkN+kACyGEEEKIetEQd3erbo5/pfMD5uM4QdZj0gEWQgghhBBnT9MQl0FrENIBFkIIIYQQZ00Byoe3N64L6QALIYQQQoj6Yff1BnhGOsBCCCGEEKJeyAiwEEIIIYRoPprQHOAGuxVyQ3He8/3wGWSNAGq8IHID8XZMKWPjitlRa922vjemrqTONKp4vojZlOJJnWn8Mc/1eL6I2eTrzGlhobG6/4V1u2Hpt98/6JPbOTe5EeAz/aCVUlu9/QZ7O6aU8dyJWZ+kzjSeeL6Iea7HawhSZ5pvPF/EPBfqTEUNcRm0htDkOsBCCCGEEKKRaiIzC+RWyEIIIYQQollpTiPAbzSDmFLGcydmY9Ac3mspY9OP15jI/tT04/ki5rlTZzSoJnIZtCZ3EpwQQgghhGh8wkJi9cC+f6lTnq//+5CcBCeEEEIIIZqwJjKues7NAVZKLVFKZSmlfqthuVJKvaKUSlZK/aqUuqCB493ojPOrUuq/Sqm+DRmvQrr+SilDKXXd2cTzNKZSaqhSartS6nel1PcNGU8p1VIp9R+l1C/OeLedZbz2Sql1SqldzvXdU02aet1vvE0pdYVSao9z+/9ezfIay1dTXqXU9c73y66UqvXb+1nG92ifP8v4PZRSm5RSJUqp+880ToX1NXg7VF0MpVS4UmqtUmqf82/rGvLW+n6caQyl1D+c69yjlBpVwzo92kZfO5v9Til1oVJqh/O9eEUppZyvv+RsJ7crpfYqpfIaMp5z2Z+UUjuddfU9L5TxVqVUdoVyvtQAbU+1+5BSqo1ytOUnlFLza1vHWcZvMm2fLyit6/TwlXOuAwwsBa6oZflooJvzcQfwWgPHOwhcqrXuAzzB2c/1cRcPpZQZeA5Yc5axPIqplGoFLACu1Fr3Aq5vyHjANGCn1rovMBR4USnlfxbxbMBMrXVP4CJgmlIqsVKa+t5vvMa5P7yKowyJwERPy+cm72/ANcAPDRXfaSlu9vl6iH8M+CvwwpnGqWQpDd8OVRfj78A3WutuwDfO5y48fD/qHMO5jglAL2eeBc5YlbndxkZiKWe+372G43M9/RlfAaC1nqG17qe17gf8C/i4IeMppboB/wAGO9vmeyvlq/eYTiucZbwQuJL6b3tq2odOAQ8B93uwjrOJ3yTaPp/Rum4PHznnOsBa6x9wHMxqMh54Szv8CLRSSkU3VDyt9X+11sedT38E4s40lifxnKYD/wayziZWHWJOAj7WWqc4059VXA/iaSDUOeIQ4kxrO4t46Vrrn53/FwK7gNhKyep1v/GyAUCy1vqA1toKLMdRnopqKl+NebXWu7TWexo4vqf7/FnF11pnaa23AKVnEafi+hq8HaohxnhgmfP/ZcBV1WT15PM4kxjjgeVa6xKt9UEg2RmrMk+20eeqK7tSqqtS6kul1E9KqfVKqR6V8zk/xzCt9SbtOMnmLaov40Tg/QaOdzvw6uljUOW22QtlbJC2hxr2Ia31Sa31Bhwd4QaL34TaPu/TgL2ODx855zrAHogFjlR4nkrVzk5DmQx80ZABlFKxwNXAwoaMU0l3oLVS6jtno3lzA8ebD/QE0oAdwD1a63qpRkqpTsD5wP8qLfLlfnO2PNn2mtLUR7nPJn59aIyfXUNtU6TWOh0cX+yAdg0Qu6YYnq7Xk21srN4ApmutL8QxyrigmjSxOMp+WpX3QSnVEegMfNvA8boD3ZVSG5VSPyqlPBlNrI8yXquU+hWYi2sHrr7aHk/3oebe9nmdom7TH3w5BaI5ngSnqnmtwT8BpdRlODrAQxo41DxgltbaUKq6ojYIC46fuoYBLYBNSqkftdZ7GyjeKGA7cDnQFVirlFqvtS44m5UqpUJwjJzfW826fLLf1BNPtr2mNPVR7rOJXx8a42fny21qqNiN8X2uN8724WLgwwpta0B1Sat5rfL7MAH4SGttNHA8C46f1ofi+PVxvVKqt9Y6rwFj/gd4X2tdopR6AxhbQzp36/J12+Pr+E1XE7m6WHPsAKcC7Ss8j8MxkthglFJ9gEXAaK11bkPGApKA5c7GKwIYo5Syaa0/bcCYqUCO1vokcFIp9QPQF2ioDvBtwLPOn96SlVIHgR7A5jNdoVLKD0fn912t9cfVJPH6flOPPNn2mtL4e5C3IePXh8b42TXUNmUqpaK11unOn1Grm450trFriuHpej3ZxsbIBORpx9zWMs55nj85n67CMYez4lS36t6HCTjOZWjoeKnAj1rrUuCgUmoPjg7xloaKWekYtwxHe13dtp12Jm2Pp/tQc2/7fKOJdICb4xSIVcDNzjMvLwLyT/+U0hCUUh1wnOhwUwOOiJbRWnfWWnfSWncCPgLuauDOL8BK4BKllEUpFQQMxDGPtqGk4BhtRikVCSQAB850Zc65xIuBXVrruTUk8+p+U8+2AN2UUp2dJwtOwFGeimoqnyd5GzJ+faiPMtS3hirvKuAW5/+34KiblZ3t+1FTjFXABKVUgFKqM46OVnVfSj3ZxkbH+avQQaXU9VB29n5frbWhnSe2aa0fdn6OhUqpi5xty81UKKNSKgFoDWzyQrxPgcuc+SNwTImosa2sj5jKdS57FGA0QNvj6T7U3Ns+72tCc4DPuRFgpdT7OH7uiVBKpQKPAH4AWuuFwGpgDI4TNIpw/XbaEPEeBtrgOCMawKbP4oLPHsSrd+5iaq13KaW+BH7FsTsv0lqfzSWr3JXxCWCpUmoHjp+PZmmtc840HjAYuAnYoZTa7nxtNtChQsx63W+8SWttU0rdjeOqIGZgidb6d6XUVOfyGstXU14ApdTVOM5kbwt8rpTarrWucumrs4nvjFNlf9BaL67P8iulooCtQBhgV0rdCySe6bQab7RDNcR4FvhAKTUZxxfF0x2ZGBz1ckxtn+nZxHC+px8AO3GclDrt9E/8SqlFwEKt9daa8jc2NZT9RuA1pdQcHJ/ncuCXarL/BccZ/C1wnPdR8dyPiThOFnQZJmugeGuAkUqpnYAB/K3iCG0DxfyrUupKHPvA6aur1GvbQy37kFLqEI567I/j5Lgn6jt+U2n7fMWX83rrQu4EJ4QQQgghzlrLoBg9qNvkOuVZ8+uTcic4IYQQQgjRVPn22r51IR1gIYQQQghx9jTSARZCCCGEEM2MD09sqwvpAAshhBBCiHrRVE6Ca46XQRNCCCGEEM2YjAALIYQQQoj60URGgKUDLIQQQgghzp4G7NIBFl6glHoIx4XLjwA5OG5XmQ/cgeNC4Mk47kJXpJRaChTjuG1wRxwX3L4FGAT8T2t9q3OdJ4BXgeHAcRw3hfgnjhtD3Ku1XqWU6gS8DQQ7N+VurfV/G7i4Qpw1qTNC1I3UGeG5pnMZNJkD3IQppZKAa4HzgWuA0xeS/lhr3V9r3RfHLYkrXpW6NXA5MAP4D/AS0As4TynVz5kmGPhOa30hUAg8CYwArgYed6bJAkZorS8AbgBeaYgyClGfpM4IUTdSZ0SdaV23h4/ICHDTNgRYqbUuBlBK/cf5em+l1JNAKyAEx20YT/uP1lo7byOcqbXe4cz7O9AJ2A5YgS+d6XcAJVrrUmeeTs7X/YD5zsbMwHGPeSEaO6kzQtSN1BlRN01kBFg6wE2bquH1pcBVWutflFK34riX+Gklzr/2Cv+ffn56fyitcJ/6snRaa7tS6nSaGUAm0BfHLwmnzrgUQniP1Bkh6kbqjPBcE5oDLFMgmrYNwDilVKBSKgT4o/P1UCBdKeWHY95WQ2gJpGut7cBNgLmB4ghRn6TOCFE3UmdEHWjQ9ro9fERGgJswrfUWpdQq4BfgMLAVx4kJDwH/c762A0dDVd8WAP9WSl0PrANONkAMIeqV1Bkh6kbqjKizJjIFQukmsqGiekqpEK31CaVUEPADcIfW+mdfb5cQjZXUGSHqRuqM8FRL/0h9cdTEOuX58sjLP2mtk9ynrF8yAtz0vaGUSgQCgWXSKAnhltQZIepG6ozwXBMZWJUOcBOntZ7k620QoimROiNE3UidEXXSRDrAchKcEEIIIYSoB3W8BrCHnWWl1BVKqT1KqWSl1N9rSddfKWUopa5zt04ZARZCCCGEEGdPA/b6vbKDUsqM466BI4BUYItSapXWemc16Z7D9ZrUNZIRYCGEEEIIUT/qfwR4AJCstT6gtbYCy4Hx1aSbDvwbxx0E3ZIOsBBCCCGEqB/13wGOBY5UeJ7qfK2MUioWx220F3q6mTIFQgghhBBC1AN9JneCi1BKba3w/A2t9RsVnld3N8LKQeYBs7TWhlI13bzQlXSAhRBCCCHE2dOg6353txw31wFOBdpXeB4HpFVKkwQsd3Z+I4AxSimb1vrTmlYqHWAhhBBCCNFYbQG6KaU6A0eBCYDLpfm01p1P/6+UWgp8VlvnF6QDLIQQQggh6kvdp0DUSmttU/+/vTu4aRgIwjA6S1qgKMrgmAM1UUZEI3RBDz8HckARQo68xljz3tGWLR8/jaydMV7q63SHU1W9JnkfY5yv9xf/9/udAAYAYI4NFmEkuVTV5ebaj+Gb5HnJOwUwAADrJdPPAd6KAAYAYI6DrEIWwAAATBETYAAA+li83GJ3AhgAgPVS00+B2IoABgBgjvsXYexCAAMAsFqqKibAAAC0kZgAAwDQiwkwAAC9HGQCPHKQ4yoAAPi/xhhvVfV452MfSZ62+J7fCGAAAFp52PsDAADgLwlgAABaEcAAALQigAEAaEUAAwDQyift+CH0ueL9wgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x360 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, axes = plt.subplots(1, 3, figsize=(13, 5))\n",
    "\n",
    "param_grid_linear = {'C': np.linspace(1, 2, 6), 'gamma': np.linspace(1, 2, 6)}\n",
    "param_grid_one_log = {'C': np.linspace(1, 2, 6), 'gamma': np.logspace(-3, 2, 6)}\n",
    "param_grid_range = {'C': np.logspace(-3, 2, 6), 'gamma': np.logspace(-7, -2, 6)}\n",
    "\n",
    "for param_grid, ax in zip([param_grid_linear, param_grid_one_log, param_grid_range], axes):\n",
    "    grid_search = GridSearchCV(SVC(), param_grid, cv=5)\n",
    "    grid_search.fit(X_train, y_train)\n",
    "    scores = grid_search.cv_results_['mean_test_score'].reshape(6, 6)\n",
    " \n",
    "    scores_image = mglearn.tools.heatmap(scores, xlabel='gamma', ylabel='C', xticklabels=param_grid['gamma'], yticklabels=param_grid['C'], cmap=\"viridis\", ax=ax)\n",
    "\n",
    "plt.colorbar(scores_image, ax=axes.tolist());"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rapid-pilot",
   "metadata": {},
   "source": [
    "### Search over spaces that are not grids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "welcome-globe",
   "metadata": {},
   "source": [
    "We can also use GridSearchCV with conditional parameters. For example, if kernel = 'linear', *gamma* is not used, but if kernel='rbf', C and gamma are used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "monthly-hawaii",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T16:15:18.316233Z",
     "start_time": "2021-03-21T16:15:17.931068Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test set score: 0.97\n",
      "Best parameters: {'C': 10, 'gamma': 0.1, 'kernel': 'rbf'}\n",
      "Best cross-validation score: 0.97\n"
     ]
    }
   ],
   "source": [
    "param_grid = [{'kernel': ['rbf'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000], 'gamma': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
    "             {'kernel': ['linear'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]}]\n",
    "\n",
    "grid_search = GridSearchCV(SVC(), param_grid, cv=5)\n",
    "grid_search.fit(X_train, y_train)\n",
    "\n",
    "print(\"Test set score: {:.2f}\".format(grid_search.score(X_test, y_test)))\n",
    "print(\"Best parameters: {}\".format(grid_search.best_params_))\n",
    "print(\"Best cross-validation score: {:.2f}\".format(grid_search.best_score_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "bigger-bible",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T16:15:56.097496Z",
     "start_time": "2021-03-21T16:15:56.050624Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>46</th>\n",
       "      <th>47</th>\n",
       "      <th>48</th>\n",
       "      <th>49</th>\n",
       "      <th>50</th>\n",
       "      <th>51</th>\n",
       "      <th>52</th>\n",
       "      <th>53</th>\n",
       "      <th>54</th>\n",
       "      <th>55</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mean_fit_time</th>\n",
       "      <td>0.00114</td>\n",
       "      <td>0.000897</td>\n",
       "      <td>0.000997</td>\n",
       "      <td>0.000801</td>\n",
       "      <td>0.001003</td>\n",
       "      <td>0.000796</td>\n",
       "      <td>0.000997</td>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.000817</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000798</td>\n",
       "      <td>0.000794</td>\n",
       "      <td>0.00102</td>\n",
       "      <td>0.000793</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000617</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000617</td>\n",
       "      <td>0.0002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std_fit_time</th>\n",
       "      <td>0.000207</td>\n",
       "      <td>0.000197</td>\n",
       "      <td>0.000003</td>\n",
       "      <td>0.000401</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>0.000398</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000509</td>\n",
       "      <td>0.00041</td>\n",
       "      <td>0.000488</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>0.000038</td>\n",
       "      <td>0.000397</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000505</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000505</td>\n",
       "      <td>0.000399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean_score_time</th>\n",
       "      <td>0.000306</td>\n",
       "      <td>0.0004</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.0002</td>\n",
       "      <td>0.000196</td>\n",
       "      <td>0.000618</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000594</td>\n",
       "      <td>0.000198</td>\n",
       "      <td>0.000598</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0004</td>\n",
       "      <td>0.000418</td>\n",
       "      <td>0.0006</td>\n",
       "      <td>0.0002</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std_score_time</th>\n",
       "      <td>0.000402</td>\n",
       "      <td>0.000489</td>\n",
       "      <td>0.000489</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000392</td>\n",
       "      <td>0.000506</td>\n",
       "      <td>0.000489</td>\n",
       "      <td>0.000485</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>0.000489</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00049</td>\n",
       "      <td>0.000513</td>\n",
       "      <td>0.00049</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000399</td>\n",
       "      <td>0.000489</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>param_C</th>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>...</td>\n",
       "      <td>1000</td>\n",
       "      <td>1000</td>\n",
       "      <td>1000</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.1</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>param_gamma</th>\n",
       "      <td>0.001</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.1</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "      <td>1000</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.1</td>\n",
       "      <td>...</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "      <td>1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>param_kernel</th>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>...</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>rbf</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "      <td>linear</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>params</th>\n",
       "      <td>{'C': 0.001, 'gamma': 0.001, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 0.01, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 0.1, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 1, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 10, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 100, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'gamma': 1000, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.01, 'gamma': 0.001, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.01, 'gamma': 0.01, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.01, 'gamma': 0.1, 'kernel': 'rbf'}</td>\n",
       "      <td>...</td>\n",
       "      <td>{'C': 1000, 'gamma': 10, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 1000, 'gamma': 100, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 1000, 'gamma': 1000, 'kernel': 'rbf'}</td>\n",
       "      <td>{'C': 0.001, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 0.01, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 0.1, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 1, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 10, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 100, 'kernel': 'linear'}</td>\n",
       "      <td>{'C': 1000, 'kernel': 'linear'}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>split0_test_score</th>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>...</td>\n",
       "      <td>0.869565</td>\n",
       "      <td>0.521739</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.869565</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.956522</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>split1_test_score</th>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>...</td>\n",
       "      <td>0.913043</td>\n",
       "      <td>0.521739</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.869565</td>\n",
       "      <td>0.913043</td>\n",
       "      <td>0.956522</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.956522</td>\n",
       "      <td>0.956522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>split2_test_score</th>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.590909</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.772727</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>split3_test_score</th>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>...</td>\n",
       "      <td>0.818182</td>\n",
       "      <td>0.590909</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.772727</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>0.909091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>split4_test_score</th>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>...</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.681818</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.954545</td>\n",
       "      <td>0.954545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean_test_score</th>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>...</td>\n",
       "      <td>0.911067</td>\n",
       "      <td>0.581423</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.366403</td>\n",
       "      <td>0.838735</td>\n",
       "      <td>0.955336</td>\n",
       "      <td>0.973123</td>\n",
       "      <td>0.972727</td>\n",
       "      <td>0.955336</td>\n",
       "      <td>0.964032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std_test_score</th>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>...</td>\n",
       "      <td>0.063488</td>\n",
       "      <td>0.058964</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.022485</td>\n",
       "      <td>0.055794</td>\n",
       "      <td>0.039794</td>\n",
       "      <td>0.021957</td>\n",
       "      <td>0.036364</td>\n",
       "      <td>0.028764</td>\n",
       "      <td>0.033918</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rank_test_score</th>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>...</td>\n",
       "      <td>23</td>\n",
       "      <td>30</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "      <td>27</td>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>16 rows × 56 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                              0   \\\n",
       "mean_fit_time                                            0.00114   \n",
       "std_fit_time                                            0.000207   \n",
       "mean_score_time                                         0.000306   \n",
       "std_score_time                                          0.000402   \n",
       "param_C                                                    0.001   \n",
       "param_gamma                                                0.001   \n",
       "param_kernel                                                 rbf   \n",
       "params             {'C': 0.001, 'gamma': 0.001, 'kernel': 'rbf'}   \n",
       "split0_test_score                                       0.347826   \n",
       "split1_test_score                                       0.347826   \n",
       "split2_test_score                                       0.363636   \n",
       "split3_test_score                                       0.363636   \n",
       "split4_test_score                                       0.409091   \n",
       "mean_test_score                                         0.366403   \n",
       "std_test_score                                          0.022485   \n",
       "rank_test_score                                               34   \n",
       "\n",
       "                                                             1   \\\n",
       "mean_fit_time                                          0.000897   \n",
       "std_fit_time                                           0.000197   \n",
       "mean_score_time                                          0.0004   \n",
       "std_score_time                                         0.000489   \n",
       "param_C                                                   0.001   \n",
       "param_gamma                                                0.01   \n",
       "param_kernel                                                rbf   \n",
       "params             {'C': 0.001, 'gamma': 0.01, 'kernel': 'rbf'}   \n",
       "split0_test_score                                      0.347826   \n",
       "split1_test_score                                      0.347826   \n",
       "split2_test_score                                      0.363636   \n",
       "split3_test_score                                      0.363636   \n",
       "split4_test_score                                      0.409091   \n",
       "mean_test_score                                        0.366403   \n",
       "std_test_score                                         0.022485   \n",
       "rank_test_score                                              34   \n",
       "\n",
       "                                                            2   \\\n",
       "mean_fit_time                                         0.000997   \n",
       "std_fit_time                                          0.000003   \n",
       "mean_score_time                                       0.000399   \n",
       "std_score_time                                        0.000489   \n",
       "param_C                                                  0.001   \n",
       "param_gamma                                                0.1   \n",
       "param_kernel                                               rbf   \n",
       "params             {'C': 0.001, 'gamma': 0.1, 'kernel': 'rbf'}   \n",
       "split0_test_score                                     0.347826   \n",
       "split1_test_score                                     0.347826   \n",
       "split2_test_score                                     0.363636   \n",
       "split3_test_score                                     0.363636   \n",
       "split4_test_score                                     0.409091   \n",
       "mean_test_score                                       0.366403   \n",
       "std_test_score                                        0.022485   \n",
       "rank_test_score                                             34   \n",
       "\n",
       "                                                          3   \\\n",
       "mean_fit_time                                       0.000801   \n",
       "std_fit_time                                        0.000401   \n",
       "mean_score_time                                       0.0002   \n",
       "std_score_time                                      0.000399   \n",
       "param_C                                                0.001   \n",
       "param_gamma                                                1   \n",
       "param_kernel                                             rbf   \n",
       "params             {'C': 0.001, 'gamma': 1, 'kernel': 'rbf'}   \n",
       "split0_test_score                                   0.347826   \n",
       "split1_test_score                                   0.347826   \n",
       "split2_test_score                                   0.363636   \n",
       "split3_test_score                                   0.363636   \n",
       "split4_test_score                                   0.409091   \n",
       "mean_test_score                                     0.366403   \n",
       "std_test_score                                      0.022485   \n",
       "rank_test_score                                           34   \n",
       "\n",
       "                                                           4   \\\n",
       "mean_fit_time                                        0.001003   \n",
       "std_fit_time                                         0.000008   \n",
       "mean_score_time                                      0.000196   \n",
       "std_score_time                                       0.000392   \n",
       "param_C                                                 0.001   \n",
       "param_gamma                                                10   \n",
       "param_kernel                                              rbf   \n",
       "params             {'C': 0.001, 'gamma': 10, 'kernel': 'rbf'}   \n",
       "split0_test_score                                    0.347826   \n",
       "split1_test_score                                    0.347826   \n",
       "split2_test_score                                    0.363636   \n",
       "split3_test_score                                    0.363636   \n",
       "split4_test_score                                    0.409091   \n",
       "mean_test_score                                      0.366403   \n",
       "std_test_score                                       0.022485   \n",
       "rank_test_score                                            34   \n",
       "\n",
       "                                                            5   \\\n",
       "mean_fit_time                                         0.000796   \n",
       "std_fit_time                                          0.000398   \n",
       "mean_score_time                                       0.000618   \n",
       "std_score_time                                        0.000506   \n",
       "param_C                                                  0.001   \n",
       "param_gamma                                                100   \n",
       "param_kernel                                               rbf   \n",
       "params             {'C': 0.001, 'gamma': 100, 'kernel': 'rbf'}   \n",
       "split0_test_score                                     0.347826   \n",
       "split1_test_score                                     0.347826   \n",
       "split2_test_score                                     0.363636   \n",
       "split3_test_score                                     0.363636   \n",
       "split4_test_score                                     0.409091   \n",
       "mean_test_score                                       0.366403   \n",
       "std_test_score                                        0.022485   \n",
       "rank_test_score                                             34   \n",
       "\n",
       "                                                             6   \\\n",
       "mean_fit_time                                          0.000997   \n",
       "std_fit_time                                                0.0   \n",
       "mean_score_time                                        0.000399   \n",
       "std_score_time                                         0.000489   \n",
       "param_C                                                   0.001   \n",
       "param_gamma                                                1000   \n",
       "param_kernel                                                rbf   \n",
       "params             {'C': 0.001, 'gamma': 1000, 'kernel': 'rbf'}   \n",
       "split0_test_score                                      0.347826   \n",
       "split1_test_score                                      0.347826   \n",
       "split2_test_score                                      0.363636   \n",
       "split3_test_score                                      0.363636   \n",
       "split4_test_score                                      0.409091   \n",
       "mean_test_score                                        0.366403   \n",
       "std_test_score                                         0.022485   \n",
       "rank_test_score                                              34   \n",
       "\n",
       "                                                             7   \\\n",
       "mean_fit_time                                          0.000622   \n",
       "std_fit_time                                           0.000509   \n",
       "mean_score_time                                        0.000594   \n",
       "std_score_time                                         0.000485   \n",
       "param_C                                                    0.01   \n",
       "param_gamma                                               0.001   \n",
       "param_kernel                                                rbf   \n",
       "params             {'C': 0.01, 'gamma': 0.001, 'kernel': 'rbf'}   \n",
       "split0_test_score                                      0.347826   \n",
       "split1_test_score                                      0.347826   \n",
       "split2_test_score                                      0.363636   \n",
       "split3_test_score                                      0.363636   \n",
       "split4_test_score                                      0.409091   \n",
       "mean_test_score                                        0.366403   \n",
       "std_test_score                                         0.022485   \n",
       "rank_test_score                                              34   \n",
       "\n",
       "                                                            8   \\\n",
       "mean_fit_time                                         0.000817   \n",
       "std_fit_time                                           0.00041   \n",
       "mean_score_time                                       0.000198   \n",
       "std_score_time                                        0.000395   \n",
       "param_C                                                   0.01   \n",
       "param_gamma                                               0.01   \n",
       "param_kernel                                               rbf   \n",
       "params             {'C': 0.01, 'gamma': 0.01, 'kernel': 'rbf'}   \n",
       "split0_test_score                                     0.347826   \n",
       "split1_test_score                                     0.347826   \n",
       "split2_test_score                                     0.363636   \n",
       "split3_test_score                                     0.363636   \n",
       "split4_test_score                                     0.409091   \n",
       "mean_test_score                                       0.366403   \n",
       "std_test_score                                        0.022485   \n",
       "rank_test_score                                             34   \n",
       "\n",
       "                                                           9   ...  \\\n",
       "mean_fit_time                                        0.000399  ...   \n",
       "std_fit_time                                         0.000488  ...   \n",
       "mean_score_time                                      0.000598  ...   \n",
       "std_score_time                                       0.000489  ...   \n",
       "param_C                                                  0.01  ...   \n",
       "param_gamma                                               0.1  ...   \n",
       "param_kernel                                              rbf  ...   \n",
       "params             {'C': 0.01, 'gamma': 0.1, 'kernel': 'rbf'}  ...   \n",
       "split0_test_score                                    0.347826  ...   \n",
       "split1_test_score                                    0.347826  ...   \n",
       "split2_test_score                                    0.363636  ...   \n",
       "split3_test_score                                    0.363636  ...   \n",
       "split4_test_score                                    0.409091  ...   \n",
       "mean_test_score                                      0.366403  ...   \n",
       "std_test_score                                       0.022485  ...   \n",
       "rank_test_score                                            34  ...   \n",
       "\n",
       "                                                          46  \\\n",
       "mean_fit_time                                       0.000798   \n",
       "std_fit_time                                        0.000399   \n",
       "mean_score_time                                       0.0004   \n",
       "std_score_time                                       0.00049   \n",
       "param_C                                                 1000   \n",
       "param_gamma                                               10   \n",
       "param_kernel                                             rbf   \n",
       "params             {'C': 1000, 'gamma': 10, 'kernel': 'rbf'}   \n",
       "split0_test_score                                   0.869565   \n",
       "split1_test_score                                   0.913043   \n",
       "split2_test_score                                        1.0   \n",
       "split3_test_score                                   0.818182   \n",
       "split4_test_score                                   0.954545   \n",
       "mean_test_score                                     0.911067   \n",
       "std_test_score                                      0.063488   \n",
       "rank_test_score                                           23   \n",
       "\n",
       "                                                           47  \\\n",
       "mean_fit_time                                        0.000794   \n",
       "std_fit_time                                         0.000397   \n",
       "mean_score_time                                      0.000418   \n",
       "std_score_time                                       0.000513   \n",
       "param_C                                                  1000   \n",
       "param_gamma                                               100   \n",
       "param_kernel                                              rbf   \n",
       "params             {'C': 1000, 'gamma': 100, 'kernel': 'rbf'}   \n",
       "split0_test_score                                    0.521739   \n",
       "split1_test_score                                    0.521739   \n",
       "split2_test_score                                    0.590909   \n",
       "split3_test_score                                    0.590909   \n",
       "split4_test_score                                    0.681818   \n",
       "mean_test_score                                      0.581423   \n",
       "std_test_score                                       0.058964   \n",
       "rank_test_score                                            30   \n",
       "\n",
       "                                                            48  \\\n",
       "mean_fit_time                                          0.00102   \n",
       "std_fit_time                                          0.000038   \n",
       "mean_score_time                                         0.0006   \n",
       "std_score_time                                         0.00049   \n",
       "param_C                                                   1000   \n",
       "param_gamma                                               1000   \n",
       "param_kernel                                               rbf   \n",
       "params             {'C': 1000, 'gamma': 1000, 'kernel': 'rbf'}   \n",
       "split0_test_score                                     0.347826   \n",
       "split1_test_score                                     0.347826   \n",
       "split2_test_score                                     0.363636   \n",
       "split3_test_score                                     0.363636   \n",
       "split4_test_score                                     0.409091   \n",
       "mean_test_score                                       0.366403   \n",
       "std_test_score                                        0.022485   \n",
       "rank_test_score                                             34   \n",
       "\n",
       "                                                 49  \\\n",
       "mean_fit_time                              0.000793   \n",
       "std_fit_time                               0.000397   \n",
       "mean_score_time                              0.0002   \n",
       "std_score_time                             0.000399   \n",
       "param_C                                       0.001   \n",
       "param_gamma                                     NaN   \n",
       "param_kernel                                 linear   \n",
       "params             {'C': 0.001, 'kernel': 'linear'}   \n",
       "split0_test_score                          0.347826   \n",
       "split1_test_score                          0.347826   \n",
       "split2_test_score                          0.363636   \n",
       "split3_test_score                          0.363636   \n",
       "split4_test_score                          0.409091   \n",
       "mean_test_score                            0.366403   \n",
       "std_test_score                             0.022485   \n",
       "rank_test_score                                  34   \n",
       "\n",
       "                                                50  \\\n",
       "mean_fit_time                                  0.0   \n",
       "std_fit_time                                   0.0   \n",
       "mean_score_time                           0.000199   \n",
       "std_score_time                            0.000399   \n",
       "param_C                                       0.01   \n",
       "param_gamma                                    NaN   \n",
       "param_kernel                                linear   \n",
       "params             {'C': 0.01, 'kernel': 'linear'}   \n",
       "split0_test_score                         0.869565   \n",
       "split1_test_score                         0.869565   \n",
       "split2_test_score                         0.772727   \n",
       "split3_test_score                         0.772727   \n",
       "split4_test_score                         0.909091   \n",
       "mean_test_score                           0.838735   \n",
       "std_test_score                            0.055794   \n",
       "rank_test_score                                 27   \n",
       "\n",
       "                                               51  \\\n",
       "mean_fit_time                                 0.0   \n",
       "std_fit_time                                  0.0   \n",
       "mean_score_time                               0.0   \n",
       "std_score_time                                0.0   \n",
       "param_C                                       0.1   \n",
       "param_gamma                                   NaN   \n",
       "param_kernel                               linear   \n",
       "params             {'C': 0.1, 'kernel': 'linear'}   \n",
       "split0_test_score                             1.0   \n",
       "split1_test_score                        0.913043   \n",
       "split2_test_score                             1.0   \n",
       "split3_test_score                        0.909091   \n",
       "split4_test_score                        0.954545   \n",
       "mean_test_score                          0.955336   \n",
       "std_test_score                           0.039794   \n",
       "rank_test_score                                11   \n",
       "\n",
       "                                             52  \\\n",
       "mean_fit_time                          0.000617   \n",
       "std_fit_time                           0.000505   \n",
       "mean_score_time                        0.000199   \n",
       "std_score_time                         0.000399   \n",
       "param_C                                       1   \n",
       "param_gamma                                 NaN   \n",
       "param_kernel                             linear   \n",
       "params             {'C': 1, 'kernel': 'linear'}   \n",
       "split0_test_score                           1.0   \n",
       "split1_test_score                      0.956522   \n",
       "split2_test_score                           1.0   \n",
       "split3_test_score                      0.954545   \n",
       "split4_test_score                      0.954545   \n",
       "mean_test_score                        0.973123   \n",
       "std_test_score                         0.021957   \n",
       "rank_test_score                               1   \n",
       "\n",
       "                                              53  \\\n",
       "mean_fit_time                                0.0   \n",
       "std_fit_time                                 0.0   \n",
       "mean_score_time                         0.000399   \n",
       "std_score_time                          0.000489   \n",
       "param_C                                       10   \n",
       "param_gamma                                  NaN   \n",
       "param_kernel                              linear   \n",
       "params             {'C': 10, 'kernel': 'linear'}   \n",
       "split0_test_score                            1.0   \n",
       "split1_test_score                            1.0   \n",
       "split2_test_score                            1.0   \n",
       "split3_test_score                       0.909091   \n",
       "split4_test_score                       0.954545   \n",
       "mean_test_score                         0.972727   \n",
       "std_test_score                          0.036364   \n",
       "rank_test_score                                3   \n",
       "\n",
       "                                               54  \\\n",
       "mean_fit_time                            0.000617   \n",
       "std_fit_time                             0.000505   \n",
       "mean_score_time                               0.0   \n",
       "std_score_time                                0.0   \n",
       "param_C                                       100   \n",
       "param_gamma                                   NaN   \n",
       "param_kernel                               linear   \n",
       "params             {'C': 100, 'kernel': 'linear'}   \n",
       "split0_test_score                        0.956522   \n",
       "split1_test_score                        0.956522   \n",
       "split2_test_score                             1.0   \n",
       "split3_test_score                        0.909091   \n",
       "split4_test_score                        0.954545   \n",
       "mean_test_score                          0.955336   \n",
       "std_test_score                           0.028764   \n",
       "rank_test_score                                11   \n",
       "\n",
       "                                                55  \n",
       "mean_fit_time                               0.0002  \n",
       "std_fit_time                              0.000399  \n",
       "mean_score_time                           0.000199  \n",
       "std_score_time                            0.000399  \n",
       "param_C                                       1000  \n",
       "param_gamma                                    NaN  \n",
       "param_kernel                                linear  \n",
       "params             {'C': 1000, 'kernel': 'linear'}  \n",
       "split0_test_score                              1.0  \n",
       "split1_test_score                         0.956522  \n",
       "split2_test_score                              1.0  \n",
       "split3_test_score                         0.909091  \n",
       "split4_test_score                         0.954545  \n",
       "mean_test_score                           0.964032  \n",
       "std_test_score                            0.033918  \n",
       "rank_test_score                                  6  \n",
       "\n",
       "[16 rows x 56 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "results = pd.DataFrame(grid_search.cv_results_)\n",
    "display(results.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "varied-harris",
   "metadata": {},
   "source": [
    "## Nested cross-validation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "divided-publisher",
   "metadata": {},
   "source": [
    "Up to now, we splitted our data in 3 fixed parts (train, test, validation). This can make ours results unstable and dependent of this single split."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "disturbed-casino",
   "metadata": {},
   "source": [
    "To solve this problem we can apply the nested cross-validation in which we run a cross-validation for differents train, tests splits. To implement this, we can simply pass GridSearchCV as the model for the cross_val_score function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "unexpected-colleague",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-21T16:29:13.673404Z",
     "start_time": "2021-03-21T16:29:12.094434Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores:  [0.96666667 1.         0.9        0.96666667 1.        ]\n",
      "Mean cross-validation score:  0.9666666666666668\n"
     ]
    }
   ],
   "source": [
    "scores = cross_val_score(GridSearchCV(SVC(), param_grid, cv=5, n_jobs=-1), iris.data, iris.target, cv=5)\n",
    "\n",
    "print(\"Cross-validation scores: \", scores)\n",
    "print(\"Mean cross-validation score: \", scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "worse-nightmare",
   "metadata": {},
   "source": [
    "As you may expect, this is a very expansive procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "stone-health",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}