Bandit Feedback - Likelihood based feature engineering logistic regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Likelihood based models\n",
    "\n",
    "This notebook will outline the likelihood based approach to training on Bandit feedback.\n",
    "\n",
    "Although before proceeding we will study the output of the simmulator in a little more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.random.mtrand import RandomState\n",
    "from recogym import Configuration\n",
    "from recogym.agents import Agent\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from recogym import verify_agents\n",
    "from recogym.agents import OrganicUserEventCounterAgent, organic_user_count_args\n",
    "from recogym.evaluate_agent import verify_agents, plot_verify_agents\n",
    "\n",
    "\n",
    "\n",
    "import gym, recogym\n",
    "from copy import deepcopy\n",
    "from recogym import env_1_args\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "plt.rcParams['figure.figsize'] = [6, 3]\n",
    "\n",
    "ABTestNumberOfUsers=5000\n",
    "NumberOfProducts=10\n",
    "NumberOfSamples = 20\n",
    "env_1_args['phi_var']=0.0\n",
    "env_1_args['number_of_flips']=0\n",
    "env_1_args['sigma_mu_organic'] = 0.0\n",
    "env_1_args['sigma_omega']=1\n",
    "env_1_args['random_seed'] = 42\n",
    "env_1_args['num_products'] = NumberOfProducts\n",
    "env_1_args['K'] = 5\n",
    "env_1_args['number_of_flips'] = 5\n",
    "\n",
    "env = gym.make('reco-gym-v1')\n",
    "env.init_gym(env_1_args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = deepcopy(env).generate_logs(ABTestNumberOfUsers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Turn Data into Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are going to build a _Logistic Regression_ model.\n",
    "\n",
    "The model will predict _the probability of the click_ for the following data:\n",
    "* _`Views`_ is a total amount of views of a particular _`Product`_ shown during _Organic_ _`Events`_ **before** a _Bandit_ _`Event`_.\n",
    "* _`Action`_ is a proposed _`Product`_ at a _Bandit_ _`Event`_.\n",
    "\n",
    "For example, assume that we have _`10`_ products. In _Organic_ _`Events`_, these products  were shown to a user as follows:\n",
    "<table>\n",
    "    <tr>\n",
    "        <th>Product ID</th>\n",
    "        <th>Views</th>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>0</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>1</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>2</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>3</td>\n",
    "        <td>7</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>4</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>5</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>6</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>7</td>\n",
    "        <td>8</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>8</td>\n",
    "        <td>11</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>9</td>\n",
    "        <td>0</td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "When we want to know the probability of the click for _`Product`_ = _`8`_ with available amounts of _`Views`_, the input data for the model will be:\n",
    "\n",
    "_`0 0 0 7 0 0 0 0 8 11 0`_ _**`8`**_\n",
    "\n",
    "The first 10 numbers are _`Views`_ of _`Products`_ (see above), the latest one is the _`Action`_.\n",
    "\n",
    "The output will be two numbers:\n",
    "* $0^{th}$ index: $1 - \\mathbb{P}_c(P=p|V)$.\n",
    "* $1^{st}$ index: $\\mathbb{P}_c(P=p|V)$.\n",
    "\n",
    "Here, $\\mathbb{P}_c(P=p|V)$ is the probability of the click for a _`Product`_ $p$, provided that we have _`Views`_ $V$.\n",
    "\n",
    "\n",
    "In all following models, an _`Action`_ will not be used as a number, but it will be decoded as a _vector_.\n",
    "In our current example, the _`Action`_ is _`8`_. Thus, it is encoded as:\n",
    "\n",
    "_`0 0 0 0 0 0 0 0`_ _**`1`**_ _`0`_\n",
    "\n",
    "Here,\n",
    "* Vector of _`Actions`_ has a size that is equal to the _*number of `Products`*_ i.e. _`10`_.\n",
    "* _`Action`_ _`8`_ is marked as _`1`_ (_`Action`_ starts with _`0`_)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import numpy as np\n",
    "\n",
    "def build_train_data(data):\n",
    "    \"\"\"\n",
    "    Build Train Data\n",
    "\n",
    "        Parameters:\n",
    "            data: offline experiment logs\n",
    "                the data contains both Organic and Bandit Events\n",
    "\n",
    "        Returns:\n",
    "            :(features, outs)\n",
    "    \"\"\"\n",
    "    num_products = int(data.v.max() + 1)\n",
    "    number_of_users = int(data.u.max()) + 1\n",
    "\n",
    "    history = []\n",
    "    actions = []\n",
    "    outs = []\n",
    "\n",
    "    for user_id in range(number_of_users):\n",
    "        views = np.zeros((0, num_products))\n",
    "        for _, user_datum in data[data['u'] == user_id].iterrows():\n",
    "            if user_datum['z'] == 'organic':\n",
    "                assert (math.isnan(user_datum['a']))\n",
    "                assert (math.isnan(user_datum['c']))\n",
    "                assert (not math.isnan(user_datum['v']))\n",
    "\n",
    "                view = int(user_datum['v'])\n",
    "\n",
    "                tmp_view = np.zeros(num_products)\n",
    "\n",
    "                tmp_view[view] = 1\n",
    "\n",
    "                # Append the latest view at the beginning of all views.\n",
    "                views = np.append(tmp_view[np.newaxis, :], views, axis = 0)\n",
    "            else:\n",
    "                assert (user_datum['z'] == 'bandit')\n",
    "                assert (not math.isnan(user_datum['a']))\n",
    "                assert (not math.isnan(user_datum['c']))\n",
    "                assert (math.isnan(user_datum['v']))\n",
    "\n",
    "                action = int(user_datum['a'])\n",
    "                action_flags = np.zeros(num_products, dtype = np.int8)\n",
    "                action_flags[int(action)] = 1\n",
    "\n",
    "                click = int(user_datum['c'])\n",
    "\n",
    "                history.append(views.sum(0))\n",
    "                actions.append(action_flags)\n",
    "                outs.append(click)\n",
    "\n",
    "    return np.array(outs), history, actions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "clicks, history, actions = build_train_data(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>c</th>\n",
       "      <th>ps</th>\n",
       "      <th>ps-a</th>\n",
       "      <th>t</th>\n",
       "      <th>u</th>\n",
       "      <th>v</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>13</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>17</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>19</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>20</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "      <td>21</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>organic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>bandit</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      a    c   ps                                               ps-a   t  u  \\\n",
       "0   NaN  NaN  NaN                                               None   0  0   \n",
       "1   3.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   1  0   \n",
       "2   4.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   2  0   \n",
       "3   5.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   3  0   \n",
       "4   NaN  NaN  NaN                                               None   0  1   \n",
       "5   2.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   1  1   \n",
       "6   8.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   2  1   \n",
       "7   4.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...   3  1   \n",
       "8   NaN  NaN  NaN                                               None   4  1   \n",
       "9   NaN  NaN  NaN                                               None   5  1   \n",
       "10  NaN  NaN  NaN                                               None   6  1   \n",
       "11  NaN  NaN  NaN                                               None   7  1   \n",
       "12  NaN  NaN  NaN                                               None   8  1   \n",
       "13  NaN  NaN  NaN                                               None   9  1   \n",
       "14  NaN  NaN  NaN                                               None  10  1   \n",
       "15  NaN  NaN  NaN                                               None  11  1   \n",
       "16  NaN  NaN  NaN                                               None  12  1   \n",
       "17  NaN  NaN  NaN                                               None  13  1   \n",
       "18  NaN  NaN  NaN                                               None  14  1   \n",
       "19  NaN  NaN  NaN                                               None  15  1   \n",
       "20  NaN  NaN  NaN                                               None  16  1   \n",
       "21  NaN  NaN  NaN                                               None  17  1   \n",
       "22  NaN  NaN  NaN                                               None  18  1   \n",
       "23  NaN  NaN  NaN                                               None  19  1   \n",
       "24  NaN  NaN  NaN                                               None  20  1   \n",
       "25  NaN  NaN  NaN                                               None  21  1   \n",
       "26  3.0  0.0  0.1  [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, ...  22  1   \n",
       "\n",
       "      v        z  \n",
       "0   0.0  organic  \n",
       "1   NaN   bandit  \n",
       "2   NaN   bandit  \n",
       "3   NaN   bandit  \n",
       "4   1.0  organic  \n",
       "5   NaN   bandit  \n",
       "6   NaN   bandit  \n",
       "7   NaN   bandit  \n",
       "8   6.0  organic  \n",
       "9   6.0  organic  \n",
       "10  2.0  organic  \n",
       "11  6.0  organic  \n",
       "12  2.0  organic  \n",
       "13  7.0  organic  \n",
       "14  6.0  organic  \n",
       "15  6.0  organic  \n",
       "16  6.0  organic  \n",
       "17  6.0  organic  \n",
       "18  6.0  organic  \n",
       "19  1.0  organic  \n",
       "20  6.0  organic  \n",
       "21  7.0  organic  \n",
       "22  7.0  organic  \n",
       "23  7.0  organic  \n",
       "24  6.0  organic  \n",
       "25  6.0  organic  \n",
       "26  NaN   bandit  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[0:27]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.]),\n",
       " array([ 0.,  2.,  2.,  0.,  0.,  0., 11.,  4.,  0.,  0.]),\n",
       " array([ 0.,  2.,  2.,  0.,  0.,  0., 11.,  4.,  0.,  0.])]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history[0:8]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0, 0, 0, 1, 0, 0, 0, 0, 0, 0], dtype=int8),\n",
       " array([0, 0, 0, 0, 1, 0, 0, 0, 0, 0], dtype=int8),\n",
       " array([0, 0, 0, 0, 0, 1, 0, 0, 0, 0], dtype=int8),\n",
       " array([0, 0, 1, 0, 0, 0, 0, 0, 0, 0], dtype=int8),\n",
       " array([0, 0, 0, 0, 0, 0, 0, 0, 1, 0], dtype=int8),\n",
       " array([0, 0, 0, 0, 1, 0, 0, 0, 0, 0], dtype=int8),\n",
       " array([0, 0, 0, 1, 0, 0, 0, 0, 0, 0], dtype=int8),\n",
       " array([1, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int8)]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actions[0:8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at the data and see how it maps into the features - which is the combination of the history and the actions and the label which is clicks.  Note that only the bandit events correspond to records in the training data.\n",
    "\n",
    "In order to do personalisation it is necessary to cross the action and history features. _Why_?  We do the simplest possible cross an element wise kronecker product.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from recogym.agents import FeatureProvider\n",
    "\n",
    "class CrossFeatureProvider(FeatureProvider):\n",
    "    \"\"\"Feature provider as an abstract class that defined interface of setting/getting features\"\"\"\n",
    "\n",
    "    def __init__(self, config):\n",
    "        super(CrossFeatureProvider, self).__init__(config)\n",
    "        self.feature_data = None\n",
    "\n",
    "    def observe(self, observation):\n",
    "        \"\"\"Consider an Organic Event for a particular user\"\"\"\n",
    "        for session in observation.sessions():\n",
    "            self.feature_data[session['v']] += 1\n",
    "\n",
    "    def features(self, observation):\n",
    "        \"\"\"Provide feature values adjusted to a particular feature set\"\"\"\n",
    "        return self.feature_data\n",
    "\n",
    "    def reset(self):\n",
    "        self.feature_data = np.zeros((self.config.num_products))\n",
    "\n",
    "\n",
    "class ModelBasedAgent(Agent):\n",
    "    def __init__(self, env, feature_provider, model):\n",
    "        # Set environment as an attribute of Agent.\n",
    "        self.env = env\n",
    "        self.feature_provider = feature_provider\n",
    "        self.model = model\n",
    "        self.reset()\n",
    "\n",
    "    def act(self, observation, reward, done):\n",
    "        \"\"\"Act method returns an action based on current observation and past history\"\"\"\n",
    "        self.feature_provider.observe(observation)\n",
    "        cross_features = np.kron(np.eye(10),self.feature_provider.features(observation))\n",
    "        prob = self.model.predict_proba(cross_features)[:, 1]\n",
    "        action = np.argmax(prob)\n",
    "        prob = np.zeros_like(prob)\n",
    "        prob[action] = 1.0\n",
    "        return {\n",
    "            **super().act(observation, reward, done),\n",
    "            **{\n",
    "                'a': action,\n",
    "                'ps': 1.,\n",
    "                'ps-a': prob,\n",
    "            }\n",
    "        }\n",
    "\n",
    "    def reset(self):\n",
    "        self.feature_provider.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_history_agent(env_args, data):\n",
    "    outs, history, actions = build_train_data(data)\n",
    "    features = np.vstack([np.kron(aa,hh) for hh, aa in zip(history, actions)])\n",
    "\n",
    "    config = Configuration(env_args)\n",
    "    logreg = LogisticRegression(\n",
    "        solver = 'lbfgs',\n",
    "        max_iter = 5000,\n",
    "        random_state = config.random_seed\n",
    "    )\n",
    "\n",
    "    log_reg_fit = logreg.fit(features, outs)\n",
    "\n",
    "    return ModelBasedAgent(\n",
    "        config,\n",
    "        CrossFeatureProvider(config),\n",
    "        log_reg_fit\n",
    "    )\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "likelihood_logreg =  build_history_agent(env_1_args, data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "organic_counter_agent = OrganicUserEventCounterAgent(Configuration({\n",
    "            **organic_user_count_args,\n",
    "            **env_1_args,\n",
    "            'select_randomly': True,\n",
    "        }))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = verify_agents(env, 5000, {'likelihood logreg': likelihood_logreg, 'organic count': organic_counter_agent})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwUAAAKjCAYAAABMV6M+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xm8JFV9///XB1BkHTYBv0EZULaIQgBZHJHBSQAliQQhagwKir8YRQaXROICA4mG5JsvCIiab1RQjEJiCP4SVNABFYewCCFRMzDDMgqyyQw7wzqf7x912ml6uu/te2/PvdP3vJ6PRz9quurUqVPV3XfqXVWnKjITSZIkSfVaa6obIEmSJGlqGQokSZKkyhkKJEmSpMoZCiRJkqTKGQokSZKkyhkKJEmSpMoZCiRJkqTKGQokSZKkyhkKJEmSpMoZCiRJkqTKGQokSZKkyhkKJEmSpMoZCiRJkqTKGQokTVsRsSQiMiJmT3VbhklEvDUi/iMiHinbb9ptw4iYXdZrSY/pz4+IT0TEwoh4orUdOsocFBHzI+LBiFhRyhw9Ge2XpEEzFEhTICLWj4g/jYh/i4hfRMTjEfFYRNweEd+IiD+OiPVK2dltO2Zjfc0rdczsMf3piLg3Ir4bEcdGxDoTWKclY2jX7hPcfrtHxLzptgNWPqd5EXHCFLbhbcDXgH2BdYF7y+upqWpTu4g4r8f3eGlE3BIRF0fERyNiuwku6hzgVGBn4BlWbodWO/YHvg28DtgQ+FWZvnyCy12jra7fXkTc2PZ5zhpk3WuKiDi6bLsJ/f2TVpdx7wBIGp+I+D3g/wJbt41+DFgBzCyvNwF/ExFHAU/QtjPSYasyfJjuOyOPdhn3ACt38NYHtgR+u7zeEREHZ+bjfa5ON4/1WG67pydQP8DuwMnAD4DzRih3K832m8j6TKaZNOv1c+DTU9SGViA5A/jzzHxmitoxmqeBZeXfAWwMbAa8FHgj8FcR8Q3gvZl5f5f5HwduBn7ZOSEiZgBHl7dvysyLusx/PM2BtX8Cjs7MaR0G2vT72+tbROwG7NY26u3AgkHUvYY5GjgAWALcOKUtkbrwTIE0icrRtYtpAsHNwFHAFpm5YWZuDGwCHAF8H/hfwGsz86rM3Lrbq63quT3K/F2XZhzeNn3jspxzyrTXAPMmuJp/16u9ba+fTXAZfcnMOZm5c2ZeOxnLmyZeXoZfWoMDAUD772KrzFwP2BR4PXAhkMCRwI0RsU3nzJl5bfluzOlS9040B82W9ggEsHI7nV9RIFhd3lGGX6Q5OPKHEfGCKWyPVCVDgTRJytGwz9P87r4F/FZmfjUzl7bKZOZDmfkvmXkg8BbgkdXdrsy8OzOPA75bRh21upepNdp6ZTja2Z41TmY+mJnfycy3AIfSnCX6DeAbY6yqn20wtNtpTVIuWXxbefu3NGcgNqE52yNpEhkKpMnzVzTXaP8S+KPRji5m5oXA6ZPRsOKyMtw6IjabxOUSERuVTp3Xl86tT0XEXRHx44j43xGxa1vZBM4tbw/ocn357LayXTsal2t7MyK+X96/NSKuioiHI+JXEfGvEbFLW/kXRcTZpb4nyrXrJ0bE2j3WZ8eIOCkiLi/9RJ6IpjPq1RHxoVZ/kY55lgBXlLfbdlmvo7vMs2tEfKljGQsi4j0R8by+Nj7P7XPSNvr2tmWf11F+3Yj4YERcExEPRcTyiLg5Ik6PiK3poss2f1tE/KD0BciIOKzf9vYjM78DfLi83adcttfenlU6GrfaSHOmDlb9HI5u204zS5kr2qZ/nw4R8ZqIuCAi7oyIJ8v6fq9856JL+ee0KyJeHxHfjoj7ounMfEJH+edHxHERcWVELCvL+Hn5XuzSWX+Zp9UvY15ErB0RJ0TEf0XTt2lZRPx7ROzVZb6+f3tjcAjNJYw/zsxFwD+W8e/oPcuv27NeWYeby/f/7rKtd+3xne6cf2b5Xd9c1v2RaP4GfSQiNugxT2tdZ0bESyLiH9o+29sj4u8iYuOOeVrfqwPKqHM7ttuSjvK7RcRXovl782Rp120R8Z3yWa0/2raRxiUzffnytZpfNEcrV9Bc0vDnA6w3y+voUcrNbCs7u0eZP2sr88JxtGVJmXfeGOebAfysbdnP0lwr/mzbuNPayt8DPFTGP1Xet79e3aVNszuWeXQZ/33gb8q/n6bpm9Fa5v3AjsAOwB1l3MM0nU5bZc7psU4/biuzHFja9vkncB2wUcc815X1bm2DzvV6c0f54zq20SMdbbsCWL/Pz+DFbctpzf+rtnFntpV9IXBDW7knOrbbMmDfLsto3+ZndfmsD+uzree16umj7PNp+uMk8PWOabPL+CVt495c1rfX5/Dmtn+3tv2ytnEXdSyj9d1qvR7q+B58HVirV7uAD5V/r6DpC/QMcEJb2RfRXJve/ttp/yyW01wu2Gsb/hXwHVb+lh7pmHe/jvn6/u2N4ff/z6W+D5T3m5Tv1DPA1qP83Wj/nT3Z1rZHgT9qTesx/+FlHVvzP1bWqfX+v4GtuszXmv5Gmt916+/C023TrgOe1+V71ar/oY7tdl1b2Td0tOOJtvVqvXYe63b25auf15Q3wJevGl40p8cH/ge9rc6jRyk3s63s7B5lvlumPzLOtixhfKHgpDLffTSXfKxTxj+PZof8I8C7O+Y5mj52DBk9FDxY/gOeS9mBBl4B3FSmXwRcA1wF7Famrw98jJU7a7t2We45wLuAbdvGrQv8Hk1fkq6Bgi47qj3W67C2nZE/o+mXAs1O8MHAojL97yfwnZrZY/q3WbkzfCSwdhm/F82OVJYdnS16bPNHynY7CdikTNsY2LLP9p3Xz2ffVv5rpfyd/W7rfj6HXt+ttulz27bFu4EZZfx6NDuJd5fpf9Fj2ctpdozPoeycAi8Atmn7fVxbyn4P2I+yI0oTFs5g5c7uS3tswwdodmz/EHh+mfZK4Cdl+rVd1qv1Ofa1/Uf5bDal2el9FnhR2/h/Kcv48AjzfpmVAeCP29b95cB/lHXrGgqAV9H87p+mCUa/UcavXbbjdWXeS0f4fTwAzKf8/ml+3+8s65M0Hdw75/0+o/y9Bm4rZf4N2LFt/MbA/jQ3qej62/Tla6KvKW+AL181vMp/PK2jPjHAeiccCsoOxNlt0z83zrYsaftPuvMIYvvrex3zfavM95ExLKuvHRNGDwUJnNxlvv3bpi+j7Lx2lJlfpp80xu20XdkZeYyOI/n0tzO6dtt6HdyjzEtL/U/TtrM1xu/UzFG2yyrLprkbVuso+6kjbPNPTeA7f14/n31b+b9oW2770due27rPz6Hrd6tM24Qm/CynhMkuZfajCUfLKDvkHctO4GsjLP/YUuaH7evVUebzpcxnemzDBF7TZb4926a/pMfn2Nf2H+Wz+dNSV+ffhMPL+J/0mG97Vp5x+aMu02cAd7XWocv0H5Vpf9Kj/s3a5t+rx+/jp8C6XeZt/S29vMu07zPC32uay6ha9a9ylsKXr9X9sk+BNDk2L8MHMjOntCVwUUTcU14P0/znd1yZdgPw0QnWvwHNzmGv1xYd5R8uwxdNcLnj8RTd+20soAlw0ISkB7uUmV+Gu3aZ1lNm3k5zudT6NLd3HKvZwLbATzPz0h7LuBW4muYOOrPHsYxejijDH3dbdmbeS7MjCs3R526eZXL7yjzQ9u/J6ivzJppnF3wvM/+rW4HM/A/gdpqj5Xv2qOd/j7CMd5ThmZnZ6xa/revzf6fH9Csz80dd2nY9cGd5O6bv9xi11uEfO8ZfQnPJzK4RsUeX+f6A5ja0d9BcgvUcmfkQK7+HzxERLwVm0Zwl/GK3Mpm5jOaMGPTedqdn5pNdxl9chuPZbo/ShB2Ymr+HqpzPKZDqs2mP8V+kOeU90YdUnZKZ88ZQ/ls0l1McHxGb01zu8aPMXO13XqI5ErzKcjJzRUTcD2xDc0Swm9azI7puz4j4HZrLCfam+Q9+lc7FNLeDHatXl+EOEXHPCOVmlOGLx7GMXlo7aFeMUOZymqPzO0bEBpn5WMf0W7L7cwOmk9Zn9LpRPqNWSHkxzSUv7ZYDXQNFNHfs2bu8/fuIOKdbOZqzSq36u7luhLb9kub73+vvxYRExM7APjTh+zm3fc3MJ6N5xsS7aILDDR2z/1YZLhjhIMuVPca3PpsNgTu79PWmbTqMfdu1nnsx5u2WmY9HxA+AA4FLI+Js4N9pzpg8O9b6pLHyTIE0OVq3Hd202x1HJtmBmRmZGTTPSziapn3vpI87fgxaZn6F5jrZoLk2+FvAgxHxnxFxakSsziNmd48w7dlRyrSmr3KXn4g4i+ZuTm+hudRhHZrLRFpPxW0d2e16h5NRtLbHuox8RqZ1n/dB3qnkhWW4ygO/2rSOMAernhWCpgPzZGrfOVvWs9RgtT6j9Rn5M3peW7lOSzNzRZfx0ISJ55d/bz5C/a3t3y2Qwsi3PG6dKev7LlZj1Ppb861yZL9T6+zBW2PVO2m11muk3+9dPca3Ppt1GPmzaf02e/1+em271nYb70HXY4GFNJcS/SXwnzR/Dy+J5kn3HszVamMokCbHwjJcl+bBSGuEzLw3M79M03EV4DMRMZ5LWibajj+hOd1+Ks11t0/SXFrzCWBxOeo+FCLi9cD7aULDPOBlNNceb54rHzp3Tav4OBbR+rv9zVa4G+U1b4Kr1M1EHiw12Uc8X1GGd45wmc2gtT6jM/v8jM7rUsdI26n9/+7f6mcZA1uzAYiItWgOAAAc3uXWpklzxgmaIPr6AS6+te3+q8/P5ugBLntUmXkbTWfvP6A5WLKQ5qzFG4DzgWsiYsPeNUjjZyiQJscPaDqPAfz+VDakm3Jd8Vdpjj6eMUVt+FlmnpzNg9s2oblTz09ojth9ucvRwjXVkWX4hcw8JTNv7XKJw1YTqL912dJLJlDHeLWO8o+07NbTg5Pmtq5TJiKeD7SeWNzrcpLVYXV/RktZGRqm4nswUXNY+T3pR+cZzNb3aqSziL2mtT6bQV5WN1CZ+UxmXpyZf5KZv0mzLn9GcxZiD+DkKW2gpi1DgTQJMvNOmstiAN7f+XCbXib5UqNPUu6mEhG/PYnLXUVmPpWZ/87KHewX0dyetKV1WcUadQS0aO3s/Ge3iRGxLc3Zg276Wa/WteevjIjfGHvzJqR1bfcBI3w3X1eGi7r0J5hs76a5DANW7cy6OrU+o9nR5UF1E1XOePy4vB3kUfR+DOK319rJ/zrN5V29XrNKud+N5z5QsfXbmjXC93D/HuNbn81mEbHPONo+EePadpl5T2b+HfDpMuqAgbZKKgwF0uT5OM1lMdsAX4uIES/BiIg/BD44GQ0DyMybgf+/vP34ZC23HM3tpf2pz+u2/bt1x6JNBt+iCWtdH/2KHtM/Re+dgtZ6zegxHZq7Ht1B04l0pLvTEBGD7iT6jTJ8Oc3DmzqXtxXwnvL2nwa87DGJiINZuX3+IzMvmcTF/zPNLWE3pXkeQ08T+IzOK8OjI2K31bSMbib024uIjWgujQH4p8x8cITXVTT37X8+8Na2ai6mOYDxYrrc5aocdHlP53iAzLyJ5s5cAH870hnI8sTkdXtNH4cRt11EPG+UA0Gtv4eDbJP0a4YCaZJk5o3A+2j+MzsU+M/ScezXR8AiYkZEHB4RVwAXAhtNcjNbO1EHRMRrJmmZ34uIsyLite1HVSPi5azc8bmb5lKilp+V4W9OwdG+0Xy3DP8kIt7ZCj0R8ZKI+DLNzs0DPeZdTNMJeUZEvKlbgXKU+Dia79FbI+Li9n4gZcdir4j4W5pbXg5MZl5J8wRcgC9FxBERsXZZ7p40nas3pblE48xBLrsf5fdzcER8nebM3Ho0AeqIkeccrMxcSnMHJoATI+IfImLHtnauFxH7R8TnaB6MNx5fpNm5fQFweUS8u/0MZERsHRFvK3ezmTvOZXQz0d/ekTSddx8Dut5St0PrzkS/voSo3HK3debnCxHxR60OuBHxmzS3Ex2pg/3xNAdoXgvMj4jXlH4ORMTaEfGKiDiJJpAM8kYHrW13eER0C/4vB34aESdExI6tgFB+029i5UGifrabNHZjfbCBL1++Jvai6dR7LysfUpM0d7J4uGPcEuC1o9Q14YeXdSm7gB5P8xxlviX09/Cye4A3t813Y1vbnqW5Q8zytnGPAXO6LO8HbWWWluUvAfbt0qbZHfMezSgPYOo172h10BzV/I+2tj1D29NVaTpPf7/X58bKJ7Umzb3UW+t1REe5Y2h2bFplHy/b4Zn279E4vp+teWf2mP5Cmss3WuWWd3x3lwH79bu9xtG+80o9T7V9n+4t69/++1lBE6y36FHPbFbTw8vaynyclQ/Zav02ltF8z1vjbh/rstvKbsnKB3G1fj9Ly3Lat8XJPbbhvBHqHuk7Oupvb4R6W/P+c5+f975ty9qlbfwmHd/DJ2h+L62/p28t/36yR72vbyvfmv/+8r1q33bbjvH3MbNVpsu0nVn5m32a5i5eS2huwQzNzRWyo02t/iOtcdcBG0/kN+TLV6+XZwqkSZaZF9PcpvJ9NEcz76S5fd06NP9BfAP4I2CnzPzhFDTxb8vwoIh41TjmH+3hZVvx3FskHkvTce4K4Bdt024CPgPsmpnzWdXhwGdpjoZvSPNAr22Z2J1xJiyb5zz8NnAazZHGFTQ76t8Ffi8z/3KUKt4D/DXN+q/LyvV6zh1HMvNcmjtZfZrmCOSzwMY0OxHfp9mmA7/TVWb+iuZpvB+mua79aZogtLi05eXZPJhrdXseK79Pm9PsQN1Gcwncx4DtM/PNOYXPRMjMvwJ2o7mLzGKas/Mb0Jz5uhT4c3pf+95P/ffRXF/+Npq/Jb9i5dnFm4Cv0Fxec9p4l9HDuH57EbEdK9f3opHKtrmGlbfAfUdrZDYPFJxFc9vOW2guyXuCpp/C3qy841u3Bw+Smd8GdqR52vwNNDvrm9AE3KtottmemfnzPts5qmwuXfodmrNtD9HcEnpbVvZDWkhzVuvzlFuR0vymH6IJf+8HZmXmw0irQWTmVLdBkiRpYCLiXcAXgB9k5uwpbo40FDxTIEmSpo3Sj6fVj+K7I5WVtJKhQJIkDZXScf/c0mF7gzJurYjYm+bSrFfQXHbzhalspzRMvHxIkiQNlYh4GU0/jZYHafo0tPo1PAEcmc3zTiT1wVAgSZKGSkSsT9Mp/yCau/q8kKaz8Z3A5cD/yczFvWuQ1MlQIEmSJFXOPgWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuXWmeoGTEcRcTuwMbBkipsiSZKk6W0m8HBmbjeRSgwFq8fG66233ma77LLLZlPdEEmSJE1fCxcuZPny5ROux1CweizZZZddNrv++uunuh2SJEmaxvbcc09uuOGGJROtxz4FkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLl1pnqBmhsZp54yYTmX3LaoQNqiSRJkqYLzxRIkiRJlfNMwZAZ6Uh/+1kEzwhIkiSpX54pkCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqN7BQEBHbRMSXIuKuiHgyIpZExKcjYtMx1rNZmW9JqeeuUu82XcpuHhHHRsS/RsQtEbE8Ih6KiB9FxLsiYpX1i4jnRcTciDg3Im6MiKciIiPi2ImsvyRJkjSsBvJE44h4KXAVsCXwTeAmYG9gLnBIRMzKzKV91LN5qWdH4HLgAmBn4Bjg0IjYLzNva5vlSOBzwN3AFcAvgK2Aw4EvAK+PiCMzM9vm2QD4dPn3vcA9wIvHs96SJEnSdDCoMwWfpQkEx2fmYZl5Yma+DjgD2An4ZJ/1fIomEJyemXNKPYfRhIsty3LaLQJ+H9gmM9+WmX+Rme+kCRJ3AG+iCQjtHgfeAPyvzNwa+NJYV1aSJEmaTiYcCspZgoOAJcA5HZNPBh4DjoqIDUapZ0PgqFJ+XsfkzwA/Bw6OiO1bIzPz8sz8t8xc0V44M+8BPl/ezu6Y9lRmfjsz7x515SRJkqQKDOJMwYFleFmXnfNHgAXA+sC+o9SzL7AesKDM117PCuDSjuWN5ukyfKbP8pIkSVKVBtGnYKcyXNRj+mKaMwk7AvMnWA+lnhFFxDrA28vb74xWfrwi4voek3ZeXcuUJEmSBm0QZwpmlOFDPaa3xm8ySfUAnAbsCnwrMy8drbAkSZJUs4HcfWhNEhHHAx+iuQPSUatzWZm5Z482XA/ssTqXLUmSJA3KIM4UtI7gz+gxvTX+wdVdT0QcB5wJ/A9wYGYuG2WZkiRJUvUGEQpuLsNe1/rvUIa9+goMpJ6IOAE4G/gpTSC4Z5TlSZIkSWIwoeCKMjyo8wnCEbERMIvm2QBXj1LP1cByYFaZr72etWg6K7cvr336R2ieiXAjTSC4b6wrIUmSJNVqwqEgM28FLgNmAu/rmHwKzROEz8/Mx1ojI2LniHjOHXoy81Hg/FJ+Xkc9x5X6L+14ojER8QmajsXXA3My8/6JrZEkSZJUl0F1NH4vcBVwVkTMARYC+9A8U2AR8LGO8gvLMDrGf5TmYWMfjIjdgWuBXYA3AvfREToi4h3AqcCzwJXA8RGdVbIkM8/rmO9EVt42dPcyPCYiXlP+/aPM/MKIayxJkiRNEwMJBZl5a0TsRbODfgjwBuBumk6/p2TmA33WszQi9qN5EvJhwP7AUuBc4KTMvLNjlu3KcG3ghB7V/gA4r2PcIcABHeNeXV4thgJJkiRVYWC3JM3MO4Bj+iy7yuH8tmnLgLnlNVo981j1UqN+lj97rPNIkiRJ09UgOhpLkiRJGmKGAkmSJKlyhgJJkiSpcoYCSZIkqXKGAkmSJKlyhgJJkiSpcoYCSZIkqXKGgmli0b2PjPhekiRJ6mVgDy/T1Fhwy/2cOX8x196+7DnjDzrjh+y93WbMnbMDs162xRS1TpIkScPAMwVD7MLrfsFRX7xmlUDQcu3tyzjqi9fwT9fdMcktkyRJ0jAxFAypBbfcz19c9BNW5MjlViSceNF/s+CW+yenYZIkSRo6hoIhdeb8xaMGgpYVCWfNX7x6GyRJkqShZSgYQovufaTnJUO9XHP7MjsfS5IkqStDwRAa76VAXkIkSZKkbgwFQ+jRJ56Z1PkkSZI0vRkKhtCGLxjfnWTHO58kSZKmN0PBEBrvcwd8XoEkSZK6MRQMoR232oi9t9tsTPPss91m7LjVRqupRZIkSRpmhoIhNXfODqwV/ZVdK+D4OTus3gZJkiRpaBkKhtSsl23BXx/+ilGDwVoBpx3+Si8dkiRJUk/2PB1ib37VS9hm0/U5a/5iruny3IJ9ttuM4+fsYCCQJEnSiAwFQ27Wy7Zg1su2YNG9j3DQGT/89fjLPvBa+xBIkiSpL14+NE10BgADgSRJkvplKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqZyiQJEmSKmcokCRJkipnKJAkSZIqt85UN0BjM/PESyZUbslphw6yOZIkSZoGPFMgSZIkVc4zBUPGI/2SJEkaNM8USJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUbWCiIiG0i4ksRcVdEPBkRSyLi0xGx6Rjr2azMt6TUc1epd5suZTePiGMj4l8j4paIWB4RD0XEjyLiXRHRc/0i4tUR8a2IWFbm+++IOCEi1h7P+kuSJEnDap1BVBIRLwWuArYEvgncBOwNzAUOiYhZmbm0j3o2L/XsCFwOXADsDBwDHBoR+2XmbW2zHAl8DrgbuAL4BbAVcDjwBeD1EXFkZmbHct4I/AvwBHAhsAz4PeAMYFapV5IkSarCQEIB8FmaQHB8Zp7dGhkRpwMfAD4JvKePej5FEwhOz8wPtdVzPHBmWc4hbeUXAb8PXJKZK9rKfxS4FngTTUD4l7ZpGwP/ADwLzM7MH5fxn6AJIkdExFsy84K+116SJEkaYhO+fKicJTgIWAKc0zH5ZOAx4KiI2GCUejYEjirl53VM/gzwc+DgiNi+NTIzL8/Mf2sPBGX8PcDny9vZHXUdAbwQuKAVCMo8TwAfL2//dKS2SpIkSdPJIPoUHFiGl3XZOX8EWACsD+w7Sj37AusBC8p87fWsAC7tWN5oni7DZzrGv64Mv9Nlnh8CjwOvjoh1+1yOJEmSNNQGcfnQTmW4qMf0xTRnEnYE5k+wHko9I4qIdYC3l7edO/89l5OZz0TE7cDLge2BhaMs5/oek3YerY2SJEnSmmIQZwpmlOFDPaa3xm8ySfUAnAbsCnwrMy/tmDbI5UiSJElDb1AdjdcYpVPyh2jugHTU6lxWZu7Zow3XA3uszmVLkiRJgzKIMwWtI+szekxvjX9wddcTEcfR3KXof4ADM3PZ6liOJEmSNJ0MIhTcXIa9rvXfoQx79RUYSD0RcQJwNvBTmkBwz1iXU/oibEfTOfm2zumSJEnSdDSIUHBFGR7U+QThiNiI5mFgjwNXj1LP1cByYFaZr72etWg6K7cvr336R2gePHYjTSC4b4TlXF6Gh3SZ9lqaOyVdlZlPjtJeSZIkaVqYcCjIzFuBy4CZwPs6Jp8CbACcn5mPtUZGxM4R8Zw79GTmo8D5pfy8jnqOK/Vf2vFE49ZDx04DrgfmZOb9ozT5G8D9wFsiYq+2el4A/FV5+7lR6pAkSZKmjUF1NH4vcBVwVkTMobmV5z40zxRYBHy9y4YgAAAgAElEQVSso3zrVp/RMf6jNA8b+2BE7E7zVOJdgDcC99EROiLiHcCpNE8nvhI4PqKzSpZk5nmtN5n5cES8myYcfD8iLgCW0TwZeacy/sL+V12SJEkabgMJBZl5aznqfirNZTlvAO6m6fR7SmY+0Gc9SyNiP5onIR8G7A8sBc4FTsrMOztm2a4M1wZO6FHtD4DzOpZzcUQcQBNW3gS8ALgF+CBwVmZmP+2VJEmSpoOB3ZI0M+8Ajumz7CqH89umLQPmltdo9cxj1UuN+pKZC2jCiyRJklS1QXQ0liRJkjTEDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUGFgoiYpuI+FJE3BURT0bEkoj4dERsOsZ6NivzLSn13FXq3aZH+SMi4uyIuDIiHo6IjIivjrKMjSLikxFxU0Q8EREPRMSlETFnLG2VJEmSpoN1BlFJRLwUuArYEvgmcBOwNzAXOCQiZmXm0j7q2bzUsyNwOXABsDNwDHBoROyXmbd1zPZxYDfgUeDOUn6kZWwK/Aj4TeBnwOeBDYE3At+LiGMz84v9rLckSZI0HQzqTMFnaQLB8Zl5WGaemJmvA84AdgI+2Wc9n6IJBKdn5pxSz2E04WLLspxOHyjzbAz8aR/LmEcTCC4Cds/MEzLzWODlwB3A2b3OSkiSJEnT0YRDQTlLcBCwBDinY/LJwGPAURGxwSj1bAgcVcrP65j8GeDnwMERsX37hMy8IjMXZ2b22eQ/KMOTMvOZtnruA04H1gPe2WddkiRJ0tAbxJmCA8vwssxc0T4hMx8BFgDrA/uOUs++NDvkC8p87fWsAC7tWN54bV2GnZchtY+zb4EkSZKqMYhQsFMZLuoxfXEZ7jhJ9Yzm/jLcrsu01lmInbpMkyRJkqalQXQ0nlGGD/WY3hq/ySTVM5pLgGOBUyLiLZn5LEBEvJCmfwJAX3dMiojre0wasbOzJEmStCYZyN2HhsxJwMHAEcCNETEf2IDm7kO/BF4CrOg9uyRJkjS9DCIUtI7gz+gxvTX+wUmqZ0SZeXdEvAr4BPC7wHtpLim6EDiT5jKl+/qsa89u48sZhD0m0k5JkiRpsgwiFNxchr2u9d+hDHv1FRh0PaPKzHuB48rr1yLideWf1010GZIkSdKwGERH4yvK8KCIeE59EbERMAt4HLh6lHquBpYDs8p87fWsRXPb0/blrQ5vL8OvrcZlSJIkSWuUCYeCzLwVuAyYCbyvY/IpNNfrn5+Zj7VGRsTOEfGczriZ+Shwfik/r6Oe40r9l3Z5ovGYRMRa5ZkIneOPogkFVwEXT2QZkiRJ0jAZVEfj99LsTJ8VEXOAhcA+NM8UWAR8rKP8wjKMjvEfBWYDH4yI3YFrgV1oOgHfx6qhg4g4DDisvG09g2C/iDiv/Pv+zPxw2yzrA/dGxHeBW2k6Fc8C9ivtOrLzeQuSJEnSdDaQUJCZt0bEXsCpwCHAG4C7aTrunpKZD/RZz9KI2I/mSciHAfsDS4FzaZ5AfGeX2XYH3tExbntWPnPg50B7KHgSuAB4DfA7ZdximuDy6cx8vJ+2SpIkSdPFwG5Jmpl3AMf0WbbzDEH7tGXA3PLqp655rHq50Ujlnwbe1W95SZIkabobREdjSZIkSUPMUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFVuYKEgIraJiC9FxF0R8WRELImIT0fEpmOsZ7My35JSz12l3m16lD8iIs6OiCsj4uGIyIj46ijLWDci3hcR10bE/RHxaEQsjIizImLbsbRXkiRJGnbrDKKSiHgpcBWwJfBN4CZgb2AucEhEzMrMpX3Us3mpZ0fgcuACYGfgGODQiNgvM2/rmO3jwG7Ao8CdpfxIy1gHmA/MKu38OvAk8Crg/cDbI+LVmfk/fay6JEmSNPQGdabgszSB4PjMPCwzT8zM1wFnADsBn+yznk/RBILTM3NOqecwmnCxZVlOpw+UeTYG/rSPZfwBTSCYD7w8M9+fmR/OzAOAU4EZwIf7bK8kSZI09CYcCspZgoOAJcA5HZNPBh4DjoqIDUapZ0PgqFJ+XsfkzwA/Bw6OiO3bJ2TmFZm5ODOzzya35r8kM1d0TPtmGb6wz7okSZKkoTeIMwUHluFlnTvZmfkIsABYH9h3lHr2BdYDFpT52utZAVzasbzx+lkZvj4iOtf/d8vwexNchiRJkjQ0BtGnYKcyXNRj+mKaMwk70lyyM5F6KPVMxCXARcDhwE8i4nvAU8CewGuAs1n1jIckSZI0bQ0iFMwow4d6TG+N32SS6hlRZmZEHEFzadPHgd9smzwf+FpmPtNPXRFxfY9JI3Z2liRJktYk1T2nICJeAFwIfAh4H/AimkDyBmBb4IcR8capa6EkSZI0uQZxpqB1BH9Gj+mt8Q9OUj2jORE4EpibmX/fNv7b5QzCjcCZrOx03FNm7tltfDmDsMcE2ylJkiRNikGcKbi5DHtd679DGfbqKzDoekbT6kx8ReeEzPwv4AFg2/LMBEmSJGnaG0QoaO1cH9R5N5+I2IjmmQCPA1ePUs/VwHJgVpmvvZ61aDorty9vvNYtw1VuOxoR6wKtZT81weVIkiRJQ2HCoSAzbwUuA2bSXKPf7hRgA+D8zHysNTIido6I53TGzcxHgfNL+Xkd9RxX6r+0yxONx+rKMvxoCQHt5tFcUnVd521RJUmSpOlqEH0KAN4LXAWcFRFzgIXAPjTPFFgEfKyj/MIyjI7xHwVmAx+MiN2Ba4FdgDcC97Fq6CAiDgMOK2+3LsP9IuK88u/7M7P9CcWfBH4PmAPcFBHfoZyhAPYu/57bz0pLkiRJ08FAQkFm3hoRewGnAofQ3MnnbpoOu6dk5gN91rM0IvajuV3oYcD+wFLgXOCkzLyzy2y7A+/oGLc9K59c/HPg16EgM38ZEXsAHwEOBY6hOWNyN3Ae8DeZeVM/7ZUkSZKmg8jMqW7DtBMR1++xxx57XH99r8cYSJIkSRO35557csMNN9zQ666Y/aruOQWSJEmSnstQIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVc5QIEmSJFXOUCBJkiRVzlAgSZIkVW6dqW6AJEmStKabeeIlE5p/yWmHDqglq4dnCiRJkqTKeaZAkiRJGsVIR/rbzyKs6WcEevFMgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVDlDgSRJklQ5Q4EkSZJUOUOBJEmSVLmBhYKI2CYivhQRd0XEkxGxJCI+HRGbjrGezcp8S0o9d5V6t+lR/oiIODsiroyIhyMiI+KrI9R/Xikz0mv+WNdfkiRJGlbrDKKSiHgpcBWwJfBN4CZgb2AucEhEzMrMpX3Us3mpZ0fgcuACYGfgGODQiNgvM2/rmO3jwG7Ao8CdpfxILgaW9Jh2FLA98O3R2ipJkiRNFwMJBcBnaQLB8Zl5dmtkRJwOfAD4JPCePur5FE0gOD0zP9RWz/HAmWU5h3TM8wGaMHALcABwxUgLyMyLaYLBc0TEJsCfA08B5/XRVkmSJGlamPDlQ+UswUE0R9/P6Zh8MvAYcFREbDBKPRvSHKl/DJjXMfkzwM+BgyNi+/YJmXlFZi7OzBzvOhRHAesBF2Xm/ROsS5IkSRoag+hTcGAZXpaZK9onZOYjwAJgfWDfUerZl2anfEGZr72eFcClHcsbtHeX4f9dTfVLkiRJa6RBXD60Uxku6jF9Mc2ZhB2BkTrw9lMPpZ6Bioj9gFcAizJzxMuPOua7vsek0fo1SJIkaRpYdO8jq7zfcauNpqg14zeIUDCjDB/qMb01fpNJqmc8/r8y/IfVULckSZKmmQW33M+Z8xdz7e3LnjP+oDN+yN7bbcbcOTsw62VbTFHrxm5QHY2HVkTMAP6QcXQwzsw9e9R5PbDHhBsnSZKkNc6F1/2Cv7joJ6zo0aP12tuXcdQXr+G0w1/JH77qxZPbuHEaRJ+C1hH8GT2mt8Y/OEn1jNUf0/R5sIOxJEmSRrTglvtHDAQtKxJOvOi/WXDLcOxeDiIU3FyGva7136EMe/UVGHQ9Y9XqYPz3A65XkiRJ08yZ8xePGghaViScNX/x6AXXAIMIBa2OuQdFxHPqi4iNgFnA48DVo9RzNbAcmFXma69nLZrOyu3Lm7CI2IfmwWeLMvP7g6pXkiRJ08+iex9ZpQ/BaK65fdkqnZHXRBMOBZl5K3AZMBN4X8fkU4ANgPMz87HWyIjYOSKec4eezHwUOL+Un9dRz3Gl/ku7PNF4IlodjL0NqSRJkkY03kuBhuESokF1NH4vcBVwVkTMARYC+9A8U2AR8LGO8gvLMDrGfxSYDXwwInYHrgV2Ad4I3MeqoYOIOAw4rLzdugz3i4jzyr/vz8wPd5lvY+DNwJPAl/tZSUmSJNXr0SeemdT5JtNAQkFm3hoRewGnAocAbwDuBs4ETsnMB/qsZ2l5ZsDJNDv6+wNLgXOBkzLzzi6z7Q68o2Pc9uUFzZOQVwkFwNtozkpcYAdjSZIkjWbDF4xv13m8802mgbUwM+8AjumzbOcZgvZpy4C55dVPXfNY9XKjfub7HPC5sc4nSZKkOo33uQPD8LyCQXQ0liRJkqa9HbfaiL2322xM8+yz3WZD8YRjQ4EkSZLUp7lzdmCtnte8PNdaAcfP2WH0gmsAQ4EkSZLUp1kv24K/PvwVowaDtQJOO/yVQ3HpEAywT4EkSZJUgze/6iVss+n6nDV/Mdd0eW7BPtttxvFzdhiaQACGAkmSJGnMZr1sC2a9bAsW3fsIB53xw1+Pv+wDrx2KPgSdvHxIkiRJGqfOADCMgQAMBZIkSVL1DAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5QwFkiRJUuXWmeoGSJIkSWu6mSdeMqFyS047dJDNGTjPFEiSJEmV80yBJEmSNIo1/Uj/RHmmQJIkSaqcoUCSJEmqnKFAkiRJqpyhQJIkSaqcoUCSJEmqnKFAkiRJqpyhQJIkSaqcoUCSJEmqnKFAkiRJqpyhQJIkSaqcoUCSJEmqnKFAkiRJqpyhQJIkSaqcoUCSJEmqnKFAkiRJqpyhQJIkSaqcoUCSJEmqXGTmVLdh2omIpeutt95mu+yyy1Q3RZIkSdPYwoULWb58+bLM3Hwi9RgKVoOIuB3YGFgyyYveuQxvmuTlSpIk1Wwq98FmAg9n5nYTqcRQMI1ExPUAmbnnVLdFkiSpFtNhH8w+BZIkSVLlDAWSJElS5QwFkiRJUuUMBZIkSVLlDAWSJElS5bz7kCRJklQ5zxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSZUzFEiSJEmVMxRIkiRJlTMUSJIkSX2KiJMi4rWjlNk/Ik6arDYNgqFAkiRJ6t88YPYoZV4LnLzaWzJA60x1AzQ+fabPFcDDwELgB5n51OptlSRJkoDn0eyHDQ1DwfCaB2Tb+2j7d+f4BJZGxPsz88JJaJskSVLN9gDun+pGjIWhYHgdCMwF3gB8BfgRcC+wFbA/cBRwCfA1mi/m+4GvRsRdmXnllLRYkiRpCEXE5R2jjo6I2V2Krg28GNgW+PrqbtcgRWaOXkprnIh4O3AO8OrM/EmX6bsBC4D3ZOZXI+IVwPXAtzPzjZPbWkmSpOEVEe2XAiXPvUKj3QpgKTAfmJuZv1rdbRsUQ8GQioj/BG7IzHeNUOZcYLfM3KO8/1dgVmZuOUnNlCRJmlZKQJiXmadOdVsGybsPDa+dgHtGKXNXKdeyGNhktbVIkiRp+jsGuHiqGzFo9ikYXo8A+41S5tXAo23vNyjzSZIkaRwy88tT3YbVwVAwvL4FvD0iPgV8MjMfa02IiA2Aj9PcI/crbfPsCiyZzEZKkiRNRxGxJbAXsClNB+NVZOZXuo1fE9mnYEhFxNbA1TQ93B8C/puVdx96Jc1lQr8A9svMuyPiRcCPgc9n5l9OTaslSZKGW0Q8D/g88HZ6X4ofQGZm17CwJjIUDLGIeCFwGvAWYL22ScuBC4ETM/O+qWibJEnSdBQRpwF/DtwK/CNwB/BMt7LDdKmRoWAaKIl1J2AGzROMb8rMp6e2VZIkSdNPRPwCeBz4rcxcPtXtGRT7FEwDJQD8dKrbIUmSVIEtgc9Op0AAhoKhV84SzAF2ATZs9ReIiBcAGwP3Z+aKEaqQJElS/35Bs481rficgiEWEYfQ3E3oEuD/APPaJu8O3A28edIbJkmSNH2dB7w+ImZMdUMGyVAwpCJiL5oHZyTwAeBr7dMz82rgduAPJr91kiRJ09ZpwI+A70XEgRExLc4aePnQ8PoETSeXvTLznog4uUuZ64A9JrdZkiRJ01rrZi4BfA8gIrqVy8wcmn3toWmoVjELuDgz7xmhzB3AoZPUHkmSpBpcSXOlxrRiKBheGwL3j1JmfbxETJIkaWAyc/ZUt2F1cIdxeP0SePkoZXYHbpuEtkiSJGmIGQqG17eBgyPiNd0mRsTrgVcD/z6prZIkSdLQ8YnGQyoifgP4L5pLhM4GZgJHAL8PvBZ4H/AQsFtmjnaZkSRJkvoQESf1WTRbz48aBoaCIRYRewD/BGzfNjppesPfChyemT+ZirZJkiRNRxEx0kNhWzvWQRMK1p6EJg2EoWDIRcTaNHcY2g/YnObswNXANzPzmalsmyRJ0nQTEQf0mLQJ8CrgeJoHy34+M38waQ2bIEPBkIqIlwBPjXJLUkmSJE2iiHgFcC3wlsz85lS3p1+GgiEVEc8CX87Md051WyRJkrRSRFwAbJeZ+0x1W/rl3YeG14OM/pwCSZIkTb5fALtOdSPGwlAwvK4GfmuqGyFJkqRV7AMsn+pGjIVPNB5e84ArI+LYzPzCVDdGkiSpBqVfZzfrAC8G3g28huYOkUPDPgVDqtwj99XA7wA30nRouYeVt8JqGap75EqSJK3Jyi1JR9qBDmAx8LrM/OXktGriDAVDapR75LYbqnvkSpIkrcki4jy6h4IVwAM0B2q/mZlPTma7JspQMKRGuEfuKobpHrmSJEmafIYCSZIkqXJ2NJYkSZLGISJeQ3M3yE2Ah4AbMvNHU9uq8fFMgSRJkjQGEbEn8P/au/dgO6vyjuPfn4lcGhSkoCCIhIiACnKpFmglFClaLWAVLSrFarQyKnWsFlO0TC8WL2NrO2Cr1okVRKHaAt5HEeWmiHLVKuINo4AiAsELlkue/vFu6jGc5Jycs7PfrL2/n3+S/a61Z34z5/zxPmetZ63Tgd3ue8Sv+gy+ARxbVV/uI9tcWRQ0ahad7wzG7wC+Dvw3cGprTS+SJEkbkySPAr4MPBi4GDgfuAnYHjiE7jjSVcATq+qbfeVcXxYFjUryOWBL4PHAvcD3gR8BD6M7I3cBcDXdFrElwKbAlcDSqvp5D5ElSZKal+S9wDHA0VX1wWnGjwLOBM6oqheMOt9ceaNxu55LVxScCSypql2q6oCq2oWuCDiTroL9fbpCYQWwL3BCT3klSZLGwaHA2dMVBABV9SHg3MG8ZlgUtOvNwK1V9byqWjl1oKpWVtXz6M7KfXNV/RQ4DrgOeNboo0qSJI2NbYBrZ5hz7WBeMywK2vUU4NMzzPk08FSAqroXuBBYvIFzSZIkjbMfA4+ZYc7uwC0jyDI0FgXtehDd9qB12XIw7z63brg4kiRJE+F84IgkR083mORZwJHAeSNNNU82GjcqyRXATsBeVXXjNOM70jUaX19V+w2enQE8qap2GmlYSZKkMTE4fehyYAvg88Bn6U4f2g44mO70oZ8CT/D0IW1wSZ5Pdz7uzcApwCX86vSh3wWOp9vLdmxVnZFkIXADcFFVHdVPakmSpPYleQJwGr+6p6Do7iqA7p6CF1TVZX1kmyuLgoYlOQF4A93xo782BNwDnFRVbxrM3YauyfiLVXXVSINKkiSNoSQH0p3uuCXd3QRXVtUl/aaaG4uCxiVZDDwf2JvuF/IOuvsI3l9V3+kzmyRJktpgUSBJkiTNUpLNgW2BH1bVXdOMb0q3nfvmqvrlqPPNlacPSZIkSbN3El3fwBZrGV9Ed0/BiSNLNAQWBY1LcnSS85L8JMk9SW5N8um1HZMlSZKkefkD4Lyqmvao98Hz84A/HGmqebIoaFQ6pwNnAIfQ3VnwY7p7CZ4MnDE4glSSJEnDszNw3QxzrhvMa4ZFQbteStdgfAVwKLBZVW0PbDb4fDlwdJLj+osoSZI0dh4IrJ5hTtG9kzXDRuNGJbmM7h6Cx1bVndOMbw58Fbi1qp4w6nySJEnjKMlXgDur6onrmPMlYIuq2mN0yebHlYJ2PQY4e7qCAGDw/BygmV9GSZKkBnwY2G9wX9T9JFlOd3fBOSNNNU8L+w6gOZt6c97azDQuSZKk9fNWui3cb0zyHOBTwA3ADsBT6O6OWgm8pbeEc+D2oUYNtg89FNhjHduH/ge4ZV3LW5IkSVo/SXYG3g/sP3g09Y+1nweOqarrRx5sHlwpaNcK4F+BCwfLVBdU1T1JFgAHAW8EHkljVaokSdLGbvDCf2CSfekKg62A24FLq+qKPrPNlSsFjUoS4H3Ac+mq09XArcDWdL0iAf6zqryvQJIkSetkUdC4JM8FXgTsA2wJrAKuBFZU1Qf6zCZJkqQ2WBRIkiRJE84jSSVJkqQJZ1EgSZIkTThPH2pEktV0DcXrq6rKn7MkSZLWypfFdlzI3IoCSZIkaZ1sNJYkSZImnD0FkiRJ0iwleXaS85M8fC3jOyT5TJJnjjrbfFgUSJIkSbP3YmCrqrpxusGquoHu7qgXjzTVPFkUSJIkSbO3J/DlGeZ8CdhrBFmGxqJAkiRJmr2tgZtnmPMTYJsRZBkaiwJJkiRp9m4Bdp1hzq7A7SPIMjQWBZIkSdLsXQIckWT36QaT7AEcCVw00lTzZFEgSZIkzd5b6e76ujjJnyd5dJJFg39fSVcMLBjMa4b3FEiSJEnrIclLgLfTvfyv6V7gZVX17tGmmh+LgkYk2Wmu362qlcPMIkmSNOkG24ReBvw2sBVdD8GlwL9V1df7zDYXFgWNSLIamMsPq6pq4bDzSJIkaXz4stiO07h/UbAYOAhYBVwF/BDYDtib7tKMC4HvjjCjJEmSGuRKQaOS7AZ8AXgP8LdVdceUsQcDf0P9XmUAAAwNSURBVAscCxxQVdf1k1KSJKltU7Zw31BV967Plu6WtnBbFDQqydnA1lW1dB1zLgRuqapnji6ZJEnS+JiyhXuPqrpuPbZ0N7WFu5mgup+DgHfMMOdi4KUjyCJJkjSu7tvCvWqNz2PFoqBdm9L1D6zL9oN5kiRJmoOq+tN1fR4XXl7WriuBo5PsM91gkv2APwauGGkqSZIkNceegkYlORT4JN0FGWfQnTT0I+BhwFLgeXRF31Or6jN95ZQkSdLGz6KgYUmOAt4JPIRf39sW4Dbgz6rqv/rIJkmSNK6SbA28CHgi3XvYdDcbV1U9eaTB5sGioHFJFgFHAvvS3U2wim7L0LlV9fM+s0mSJI2bJLsDnwO2pftD7NpUVU1XLGyULAokSZKkWUryUeBpwJuAdwHfr6p7+001fxYFYyLJg4CtgFVTLzKTJEnS8CRZBVxYVYf3nWWYPH2oYUkWJlme5FvA7cD1wG1JvjV47pGzkiRJwxXga32HGDZXChqVZBO604eW0jUZ/wC4ie5ugh3pfmEvAg6rqrv6yilJkjROknyWbmfGM/rOMkyuFLTrL4CDgY/RXbu9c1UdUFU7A7sBHwGeNJgnSZKk4fg74GlJDu47yDC5UtCoJNcM/rt3Va2eZvwBwFV0P+M9RxpOkiRpTCU5Fjic7vTHDwCX023jvp+qOm2E0ebFoqBRSX4BnFJVr13HnDcDx1fVb4wumSRJ0vhKsppu6/bU40jXfKEOjR1JaiNqu+4CtphhziLg7hFkkSRJmhQv7DvAhuBKQaOSXEjXO/C4qvrxNOPbAF8Frquqg0adT5IkSe2w0bhdp9LdpHdZkmVJdkmyeZLFSV4IfHEwfmqvKSVJkrTRc6WgYUlOBpZz/31s0O1le0tVLR9tKkmSJLXGoqBxSfYHlgH7AFsCq4ArgRVV9YU+s0mSJI2jJIuAlwFPAXYANp1mWlXVkpEGmweLAkmSJGmWkmwFXAw8BrgDeDDdH2U3ATYfTLsRuLuqFvcScg7sKZAkSZJm7/V0BcEy4CGDZ2+jOxXyQOAK4NvAHr2kmyOLgsYl2T/Ju5NcnuTbSa5I8u9JDuw7myRJ0hg6Ariwqt5TU7bcVOdS4GnA7sDr+go4FxYFDUvyBuAS4EV0PQWLgb3pKteLBo3IkiRJGp5H0N1ifJ/VTOkpqKqbgU8AR48417xYFDQqybOBE4GVwIuBXej2se0y+LwSeG2S5/QWUpIkafz8gq4QuM8qYLs15vyIrgG5GTYaN2pwedmuwJ5Vdcs04/ddXvaNqlo66nySJEnjKMlVwLVVdfTg80V0RcFuVbV68OwSYPuq2qW/pOvHlYJ2PR740HQFAcDg+QfpthNJkiRpOC4AlibJ4PNZwBLg40lenuSDwP7Ax/sKOBcL+w6gOVtIt3y1Lr/An7EkSdIwvZfu+NEdge8D7wAOAZ4BHDaYcwndKUXNcPtQo5JcAyyg2z60eprxBwBX0zXD7zXqfJIkSZMkyX7Ao4DrgS9N9362MXP7ULveT3f+7blJdp06kGQJ8CG6M3Tf30M2SZKkiVJVl1fVWVX1xdYKAnCloFlJNgE+BRxE1wF/I3ATXaPLDnQF38XAoVV1V185JUmStPGzKGhYkgcCr6G7p2DJlKFvAyuAt1bV3X1kkyRJGkdJTprFtNXAHcDXgQta+AOtRcGYSLIFsCWwqqp+1nceSZKkcZRkNTD1BTpT/r/m8wJ+AhxfVWeNIN6cWRRIkiRJs5RkKfBK4GnAaXTbtX8EPAx4EvAnwMfo+jr3BY6nu2D2kKq6qI/Ms2FRIEmSJM1SkmOBtwMHVtVXphl/PN2RpMdV1fuS7AlcDnyiqo4cbdrZsyho2KBS/UvgicBDmP40qaoq7yqQJEkagiRXAldU1bJ1zHkP8Piq2nfw+Wzgd6rqoSOKud58WWxUkqcD59DdVbAS+AZwT6+hJEmSxt9uzHxb8Y3Ac6Z8/ibw9A2WaAgsCtr1N8DdwNOr6lM9Z5EkSZoUPwUOmGHOgcDUg18WDb630fLysnY9DjjLgkCSJGmkPg4sTXJykkVTB5IsSvJGunukpq4mPI7upuONlisF7foZcGvfISRJkibMXwG/B7wWOC7JNfzq9KG9gK3otnafCJBke+BRwDt6STtLNho3KsmZwE5VdWDfWSRJkiZJkm2BNwFH0x03ep87gbOA5VV1cx/Z5sqioFFJHglcBpwC/EP5g5QkSRqpJA+kazzeku4G42ur6u5+U82NRUEjkqyY5vHOwFLge8BVwO3TzKl1HZklSZKk2UtyEvDdqjq97yzDZFHQiMGV2nNRVbVgqGEkSZImVJK7gH+uqhP6zjJMNhq3Y3HfASRJksQNwIP7DjFsFgWNqKrv9Z1BkiRJnA0ckWTzqrqz7zDD4vYhSZIkaZaSPAg4n+5o+FdX1Vd7jjQUFgWNSLLT4L83VNW9Uz7PqKpWbqBYkiRJEyXJd4BNgO0Hj34J3Ays+VJdVbVklNnmw6KgEYNG4wL2qKrrpnyeSVWV28QkSZKGIMn1zO4djKpqpifUl8V2nEb3C7hqjc+SJEkakaraue8MG4IrBZIkSdKEc6VAkiRJmqNB4/FWwKqquqPvPHP1gL4DSJIkSS1JsjDJ8iTfAm4HrgduS/KtwfPm/vDu9qFGJFkxx69WVS0bahhJkqQJlWQT4JPAUrr+zh8AN9GdRrQjEOAi4LCququvnOvLoqARg9OG5qKqasFQw0iSJE2oJMuBk4GP0t1T8M0pY0uAfwQOB15XVW/qJ+X6syhoRJJHzvW73oYsSZI0HEmuGfx376q63x9tkzwAuIruPXvPkYabh+b2O00qX+wlSZI2Co8CTpmuIACoqtVJPgEcP9pY82OjsSRJkjR7dwFbzDBnEXD3CLIMjUVB45IcnuTMJFcPOuDve75HkhOS7NBnPkmSpDFzDXBUkm2nG0yyDXAUcPVIU82TRUGj0nkvcA7wbGAJMPUq7dvommCO6SGeJEnSuDoV2Ba4LMmyJLsk2TzJ4iQvBL44GD+115TryUbjRiV5OXAKsAJ4NfAq4K+nnjSU5AK6n/FB/aSUJEkaP0lOBpbTHUl6v2HgLVW1fLSp5sdG43Yto1uWeklVVZLpfim/CTxltLEkSZLGW1WdmOTDdO9j+wBbAquAK4EVVfWFPvPNhUVBu3YD3lnrXuq5mW75SpIkSUNUVZcCl/adY1jsKWjXPcBmM8zZAfjZCLJIkiSpYRYF7foacHCSTDeYZDPgELplLEmSJGmtLAradTqwO/C2wc15/y/JAuCfgIcD/zH6aJIkSWqJpw81avDi/zHgMOAm4KfArsDZwP50BcG5VfVHvYWUJElSEywKGpZkIfB64BXA1lOGbqc7rvTvq+qePrJJkiSpHRYFY2DQV/Bo4DfpjsO6tqru7TeVJEmSWmFR0Kgkm1bV/85i3s5Vdf0IIkmSJKlRNhq364yZJiR5BHD+CLJIkiSpYRYF7Xpmkn9Z22CS7egKgh1GF0mSJEktsiho1ynAK5K8Zs2BJA8FPgssBo4ddTBJkiS1xZ6CRg2aiz8EHAkcU1VnDp5vDXwOeCzwwqo6rbeQkiRJaoJFQcOSbAp8Bvgt4Kl0txefD+wDHFdV7+oxniRJkhphUdC4wcrAJcB2wHfoCoJXVdVa+w0kSZKkqSwKxkCSnYAv0BUGJ1bVm3uOJEmSpIZYFDQiyYoZpuwO7Ap8ZI3nVVXLNkwqSZIkjQOLgkYkWT3Hr1ZVLRhqGEmSJI2VhX0H0Kwt7juAJEmSxpMrBZIkSdKE8/IySZIkacK5fagRgxOGAG6oqnunfJ5RVa3cQLEkSZI0Btw+1IhBo3EBe1TVdVM+z6SqyuJPkiRJa+XLYjtOoysCVq3xWZIkSZoXVwokSZKkCWejsSRJkjThLAokSZKkCWdPQSOSrJjjV6uqlg01jCRJksaKPQWNGJw2NBdVVQuGGkaSJEljxZWCdizuO4AkSZLGkysFkiRJ0oSz0ViSJEmacBYFkiRJ0oSzKJAkSZImnEWBJEmSNOEsCiRJkqQJZ1EgSZIkTTiLAkmSJGnCWRRIkiRJE86iQJIkSZpwFgWSJEnShLMokCRJkiacRYEkSZI04SwKJEmSpAn3f1CFWzTHt9CuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x131da1978>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 337,
       "width": 386
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_verify_agents(result)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}