06-Clustering-I-kmeans.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "hide_input": true,
    "slideshow": {
     "slide_type": "skip"
    },
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format='retina'\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from IPython.display import Image, display_html, display, Math, HTML;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# $k$-means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "\n",
    "<img src=\"figs/L6-John_Snow_memorial_and_pub.png\" alt=\"Figure\" width=\"400px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div style = \"float: left; width: 55%;\">\n",
    "    \n",
    "__1854__ saw a horrific cholera outbreak in Soho, London.   \n",
    "\n",
    "Common wisdom at the time was that disease spread by breathing \"foul air\" (miasma).  \n",
    "    \n",
    "The London sewer system had not yet reached Soho.  Most homes had cesspits under the floor.\n",
    "\n",
    "John Snow, a local physician, extensively studied the patterns of illness across Soho due to cholera.\n",
    "    \n",
    "</div>\n",
    "\n",
    "<img src=\"figs/L06-John-Snow-Portrait.png\" alt=\"Figure\" width=\"30%\" float = \"right\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In the course of his studies, his attention was drawn to one neighborhood around Broad Street.\n",
    "\n",
    "In 10 days, 500 people in the area died. \n",
    "\n",
    "In uncovering the source of this outbreak, Snow prepared this map:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-Snow-cholera-map-1.png\" alt=\"Figure\" width=\"500px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "By John Snow - Published by C.F. Cheffins, Lith, Southhampton Buildings, London, England, 1854 in Snow, John. On the Mode of Communication of Cholera, 2nd Ed, John Churchill, New Burlington Street, London, England, 1855."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "(This image was originally from <a class=\"external text\" href=\"http://en.wikipedia.org\">en.wikipedia</a>; description page is/was <a class=\"external text\" href=\"http://en.wikipedia.org/w/index.php?title=Image%3ASnow-cholera-map-1.jpg\">here</a>. Image copied from <a rel=\"nofollow\" class=\"external free\" href=\"http://matrix.msu.edu/~johnsnow/images/online_companion/chapter_images/fig12-5.jpg\">http://matrix.msu.edu/~johnsnow/images/online_companion/chapter_images/fig12-5.jpg</a>), Public Domain, <a href=\"https://commons.wikimedia.org/w/index.php?curid=2278605\">https://commons.wikimedia.org/w/index.php?curid=2278605</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-John_Snow_memorial_and_pub.png\" alt=\"Figure\" width=\"400px\">\n",
    "\n",
    "\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<a href=\"http://creativecommons.org/licenses/by-sa/2.0\" title=\"Creative Commons Attribution-Share Alike 2.0\">CC BY-SA 2.0</a>, <a href=\"https://commons.wikimedia.org/w/index.php?curid=357998\">https://commons.wikimedia.org/w/index.php?curid=357998</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "John Snow's original data is available here: http://blog.rtwilson.com/john-snows-cholera-data-in-more-formats/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Clustering is Unsupervised Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Clustering is a very important way of discovering __structure__ in data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "It is so important because it is __common__ for data to show clusters.\n",
    "\n",
    "* Locations where millionaires live\n",
    "* The number of hours people work each week\n",
    "* Demographics (\"soccer moms\", \"bored retirees\", \"unemployed millenials\", etc)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can often simplify or compress our data if we recognize the existence of clusters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Further, we can often interpret clusters by assigning them labels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "However, note that these categories or \"labels\" are assigned __after__ the fact.\n",
    "\n",
    "And, we may __not__ be able to interpret clusters or assign them labels in some cases."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "That is, clustering represents the first example we will see of __unsupervised__ learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Supervised__ methods:  Data items have labels, and we want to learn a function that correctly assigns labels to new data items.\n",
    "\n",
    "__Unsupervised__ methods:  Data items do not have labels, and we want to learn a function that extracts  important patterns from the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Applications of Clustering:__\n",
    "\n",
    "* Image Processing\n",
    "    * Cluster images based on their visual content\n",
    "    * Compress images based on color clusters\n",
    "* Web Mining\n",
    "    * Cluster groups of users based on webpage access patterns\n",
    "    * Cluster web pages based on their content\n",
    "* Bioinformatics\n",
    "    * Cluster similar proteins together (by structure or function)\n",
    "    * Cluster cell types (by gene activity)\n",
    "* And many more ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Clustering Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "When we do clustering, what problem are we trying to solve?\n",
    "\n",
    "We will answer this question informally at first. \n",
    "\n",
    "(But soon we will look at formal criteria!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Informally, a __clustering__ is:\n",
    "    \n",
    "> a grouping of data objects, such that the objects within a group are similar (or near) to one another and dissimilar (or far) from the objects in other groups.\n",
    "\n",
    "(keep in mind that if we use a distance function as a dissimilarity measure, then \"far\" implies \"different\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-clustering-1.png\" alt=\"Figure\" width=\"350px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "So we want our clustering algorithm to:\n",
    "* <font color = \"blue\">minimize</font> intra-cluster distances\n",
    "* <font color = \"red\">maximize</font> inter-cluster distances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here are the basic questions we need to ask about clustering:\n",
    "    \n",
    "* What is the right kind of <font color = \"blue\">\"similarity\"</font> to use?\n",
    "* What is a <font color=\"blue\">\"good\" partition</font> of objects?\n",
    "    * ie, how is the quality of a solution measured?\n",
    "* <font color = \"blue\">How to find</font> a good partition?\n",
    "    * are there efficient algorithms?  \n",
    "    * are there algorithms that are guaranteed to find good clusters?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now note that even with our more-formal discussion, the criteria for deciding on a \"best\" clustering can still be ambiguous."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-clustering-2.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To accommodate the ambiguity here, one approach is to seek a __hierarchical__ clustering:\n",
    "\n",
    "That is, as set of nested clusters organized in a tree.\n",
    "\n",
    "We'll discuss hierarchical cluster in an upcoming lecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For today, we'll focus on __partitional__ clustering:  \n",
    "\n",
    "in a partitional clustering, the points are divided into a set of __non-overlapping__ groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-partitional-clustering.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In a partitional clustering:\n",
    "* Each object belongs to __one__, and only one, cluster\n",
    "* The set of clusters covers all the objects\n",
    "\n",
    "We are going to assume for now that the number of clusters is given in advance.\n",
    "\n",
    "We will denote the number of clusters as $k$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The $k$-means Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Now, we are ready to state our first formalization of the clustering problem.\n",
    "\n",
    "We will assume that \n",
    "* data items are represented by points in $\\mathbb{R}^d$.   (In other words, each data item has $d$ features.)\n",
    "* $n$ points are given\n",
    "* the number of clusters $k$ is given"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__$k$-means Problem:__\n",
    "\n",
    "Find $k$ points $c_1, \\dots, c_k$ (called <font color=\"blue\"> _centers_, _centroids_,</font> or <font color = \"blue\"> _means_,</font> so that the cost\n",
    "\n",
    "$$ \\sum_{i=1}^n \\min_k \\Vert x_i-c_j\\Vert^2_2 $$\n",
    "\n",
    "is minimized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Equivalently:__ we can think in terms of the partition itself.\n",
    "\n",
    "Consider the set $X = \\{x_1, \\dots, x_n\\}$ where $x_i \\in \\mathbb{R}^n$.\n",
    "    \n",
    "Find $k$ points $c_1, \\dots, c_k$ \n",
    "\n",
    "and partition $X$ into subsets $\\{X_1, \\dots, X_k\\}$ by assigning each point $x_i$ in $X$ to its nearst cluster center,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "so that the cost\n",
    "\n",
    "$$ \\sum_{i=1}^n \\min_j \\Vert x_i-c_j\\Vert^2_2 = \\sum_{j=1}^k \\sum_{x \\in X_j} \\Vert x - c_j \\Vert^2_2 $$\n",
    "\n",
    "is minimized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We now have a <font color=\"blue\">formal</font> definition of a clustering.\n",
    "\n",
    "This is not the only definition possible, but it is an intuitive and simple one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "How hard is it to solve this problem?\n",
    "\n",
    "* $k=1$ and $k=n$ are easy special cases (why?)\n",
    "* But, this problem is __NP-hard__ if the dimension of the data is at least 2\n",
    "    * We don't expect that there is any exact, efficient algorithm in general \n",
    "\n",
    "__Nonetheless,__ there is a simple algorithm that works quite well in practice!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The $k$-means Algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "There is a \"classic\" algorithm for this problem.\n",
    "\n",
    "It was voted among the __top-10 algorithms__ in data mining!\n",
    "\n",
    "It is such a good idea that it has been independently discovered multiple times.\n",
    "\n",
    "It was first discovered by Lloyd in 1957, so it is often called Lloyd's algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "It is called the \"$k$-means algorithm\"\n",
    "\n",
    "(not to be confused with the $k$-means problem!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-top-ten-algorithms-cover.png\" alt=\"Figure\" width=\"300px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The $k$-means algorithm:\n",
    "\n",
    "1. Pick $k$ cluster centers $\\{c_1, \\dots, c_k\\}$.  These can be chosen randomly, or by some other method.\n",
    "2. For each $j$, define the cluster $X_j$ as the set of points in $X$ that are <font color=\"blue\">closest to center</font> $c_k$.  \n",
    "(Nearer to $c_k$ than to any other center.)\n",
    "3. For each $j$, let $c_j$ be <font color=\"blue\">the center of mass of cluster</font> $X_j$.  \n",
    "(In other words, $c_j$ is the mean of the vectors in $X_j$.)\n",
    "4. Repeat (ie, go to Step 2) until convergence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let's see this in practice:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-kmeans-example.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Limitations of $k$-means\n",
    "\n",
    "As you can see, $k$-means can work very well.\n",
    "\n",
    "However, we don't have any guarantees on the performance of $k$-means.\n",
    "\n",
    "In particular, there are various settings in which $k$-means can fail to do a good job."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "1. __$k$-means tries to find spherical clusters__.\n",
    "\n",
    "Because each point is assigned to its closest center, the points in a cluster are implicitly assumed to be arranged in a sphere around the center."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-kmeans-nonspherical-clusters.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "2. __$k$-means tries to find equal-sized clusters.__\n",
    "\n",
    "For the same reason, the sizes of clusters are implicitly assumed to be approximately equal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-kmeans-cluster-size.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "3. __$k$-means is sensitive to the starting cluster centers.__\n",
    "\n",
    "If the initial guess (Step 1) is a bad one, $k$-means may get \"stuck\" in a bad solution.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L06-kmeans-bad-initialization.png\" alt=\"Figure\" width=\"600px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Choosing a Good Initialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How can we avoid the kind of bad initialization we just saw?\n",
    "\n",
    "A good strategy is to pick points that are distant to each other.\n",
    "\n",
    "This strategy is called \"$k$-means++\".  \n",
    "\n",
    "It works very well in practice, and the `scikit-learn` implementation uses it by default.\n",
    "\n",
    "(We will explore it in the next lecture.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Choosing the right $k$\n",
    "\n",
    "Generally, we would say that, given some $k$, the $k$-means algorithm \"learns\" the cluster centers -- that is, the parameters of the model.\n",
    "\n",
    "But we have not yet considered how to choose the right number of clusters.\n",
    "\n",
    "That's typically __not__ something one knows in advance.\n",
    "\n",
    "As an aside:\n",
    "* This parameter ($k$) is the first example we have seen of a __hyperparameter__.\n",
    "* A hyperparameter is a parameter that must be set before the model parameters can be learned."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our basic strategy will be to\n",
    "* Iterate through different $k$ and use some criterion to decide which $k$ is most appropriate.\n",
    "* We will discuss this more in the next lecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature Scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Finally, given the tendency of $k$-means to look for spherical clusters, we should consider the scales of the various features.\n",
    "\n",
    "In fact, in general when constructing or selecting a distance metric, one needs to think carefully about the scale of the features being used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For example, consider the case where we are clustering people based on their age, income, and gender.\n",
    "\n",
    "We might use age in years, income in dollars, and assign gender to the values $\\{0, 1\\}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Thus, the following records:\n",
    "\n",
    "* Joe Smith, age 27, income USD 75,000, male\n",
    "* Eve Jones, age 45, income USD 42,000, female\n",
    "\n",
    "Would be encoded in feature space as:\n",
    "\n",
    "$$\\begin{bmatrix}27\\\\75000\\\\0\\end{bmatrix},\\begin{bmatrix}45\\\\42000\\\\1\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "What would happen if we used Euclidean distance as our dissimilarity metric in this feature space?\n",
    "\n",
    "(This is what $k$-means uses.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Clearly, the influence of income would dominate the other two features.  For example, a difference of gender is about as significant as a difference of one dollar of yearly income.\n",
    "\n",
    "We are unlikely to expose gender-based differences if we cluster using this representation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The most common way to handle this is __feature scaling.__\n",
    "\n",
    "The basic idea is to rescale each feature separately, so that its range of values is about the same as all other features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For example, one may choose to:\n",
    "    \n",
    "* shift each feature independently by subtracting the mean over all observed values\n",
    "    * This means that each feature is now centered on zero\n",
    "* then rescale each feature so that the standard deviation overall observed values is 1.\n",
    "    * This means that the feature will have about the same range of values as all the others."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For example, let's work with Bortkiewicz's famous horse-kick data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Year\n",
       "1875.0     3.0\n",
       "1876.0     5.0\n",
       "1877.0     7.0\n",
       "1878.0     9.0\n",
       "1879.0    10.0\n",
       "1880.0    18.0\n",
       "1881.0     6.0\n",
       "1882.0    14.0\n",
       "1883.0    11.0\n",
       "1884.0     9.0\n",
       "1885.0     5.0\n",
       "1886.0    11.0\n",
       "1887.0    15.0\n",
       "1888.0     6.0\n",
       "1889.0    11.0\n",
       "1890.0    17.0\n",
       "1891.0    12.0\n",
       "1892.0    15.0\n",
       "1893.0     8.0\n",
       "1894.0     4.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# source: http://www.randomservices.org/random/data/HorseKicks.html\n",
    "import pandas as pd\n",
    "df = pd.read_table('data/HorseKicks.txt',index_col='Year',dtype='float')\n",
    "counts = df.sum(axis=1)\n",
    "counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAH8CAYAAACD9VqAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAABYlAAAWJQFJUiTwAAAupUlEQVR4nO3dfZhsV10n+u8PEgLEDhAUGMNLSEwIEkQJCkwQTmBuRLxiUHFmLiJw1QySEUSiRNBLgoODA2oIb5qrGAk6d7gqMhJEHM0BkogvBxSiAULICRgCCIHQiSFCzpo/djVdp+mXOqequ1af/nyep569a7+sWrV6V9e39sva1VoLAADQhzvMuwIAAMAyAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6Mhh867AVqqqa5MclWTvnKsCAMCh7dgkX2ytPfBAV9xRAT3JUXe5y12OfvCDH3z0vCvSm8XFxSTJwsLCnGuyfWnD6WnD2dCO09OG09OG09OG05tnG1511VW59dZbD2rdnRbQ9z74wQ8+es+ePfOuR3d2796dJNm1a9dc67GdacPpacPZ0I7T04bT04bT04bTm2cbnnLKKXnf+96392DWdQ46AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEdmEtCr6per6s+r6hNVdWtV3VhV76+ql1TVPQ+wrPtW1Ruq6pNVdVtV7a2q86vqHrOoKwAA9GxWe9Cfn+TIJH+W5FVJfjfJV5Kcm+QDVXW/SQqpquOT7EnyrCR/neTXknwsyfOS/OWBhn0AANhuDptROUe11r60cmJVvSzJi5L8XJLnTFDO65LcK8lzW2uvHivnVzP8CHhZkmfPpMYAANChmexBXy2cj7x5NDxhozKq6rgkpyfZm+S1K2a/JMktSZ5eVUceZDUBAKB7s9qDvpbvHQ0/MMGyjx8N39la2zc+o7W2WFWXZwjwj0ry5+sVVFV71ph10uLiYnbv3j1BdXaWxcXFJNE2U9CG09OGs6Edp6cNp6cNp6cNpzfPNlx67YMx04BeVWcn+bokd0vyiCSPyRDOXz7B6g8aDT+yxvyrMwT0E7NBQAdgvq68/qZNLf/kY+62qeUDzNOs96CfneTeY8/fkeSZrbV/nmDdpf+2a/1XX5p+940Kaq2dstr0qtqzsLDw8F27dk1QnZ1l6Zeltjl42nB62nA2emjHZ55zyaaWv/dpuza1/B7acLvThtPThtObZxsuLCwc9Loz7Qe9tXaf1loluU+S709yXJL3V9XDZ1B8Lb3MDMoCAIAubcqNilprn26tvSXDKSn3TPLGCVZb2kO+1nHLo1YsBwAAh5xNvZNoa+26JP+Y5CFV9fUbLP7h0fDENeYv9QSz1jnqAACw7W1qQB/5xtHw9g2Wu3Q0PL2q9qtXVS0kOTXJrUneO9vqAQBAP6YO6FV1UlXdZ5XpdxjdqOheSa5orX1+NP3w0TrHjy/fWrsmyTuTHJvkrBXFnZfhTqVvbK3dMm2dAQCgV7PoxeWJSV5RVe9Ock2Sz2XoyeVxGS4S/VSSHx9b/pgkVyW5LkMYH/ecJFckuaCqnjBa7pFJTstwasuLZ1BfAADo1iwC+v9KcmGGU1AelqEbxFsyBOqLk1zQWrtxkoJaa9dU1SOSvDRD8H9SkhuSXJDkvEnLAQCA7WrqgN5auzJfe0rKesvvzXKXiavN/0SSZ01bLwAA2I624iJRAABgQgI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEemDuhVdc+q+rGqektVfbSqbq2qm6rqsqr60aqa+DWqam9VtTUen5q2rgAA0LvDZlDGU5O8PskNSS5N8vEk907y/Ul+M8l3V9VTW2ttwvJuSnL+KtNvnr6qAADQt1kE9I8keXKSS1pr+5YmVtWLkvx1kh/IENb/YMLyvtBaO3cG9QIAgG1n6lNcWmt/0Vr74/FwPpr+qSS/Pnq6a9rXAQCAnaAmP/PkIAqv+pkk/y3J+a2150+w/N4kRyT5mST3T3JLkg8keXdr7fYDeN09a8w66YQTTrjrhRdeOGlRO8bi4mKSZGFhYc412b604fS04Wz00I5XXn/TppZ/8jF329Tye2jD7U4bTk8bTm+ebXjmmWfm6quvfl9r7ZQDXXcWp7isqqoOS/Ijo6fvOIBV75Pk4hXTrq2qZ7XW3jWTygEAQKc2LaAneXmSk5O8vbX2pxOu89tJ3pPkH5IsJjkuyX9OcmaSP6mqR7fW/n6jQtb6pVJVexYWFh6+a9euCauzc+zevTtJom0OnjacnjacjR7a8ZnnXLKp5e992q5NLb+HNtzutOH0tOH05tmG0+y135SAXlXPTfKCJB9K8vRJ12utnbdi0pVJnl1VN4/KOzfJU2ZUTQAA6M7Mb1RUVWcleVWSf0xyWmvtxhkUu3Sx6WNnUBYAAHRrpgG9qn4qyWsy7Pk+bdSTyyx8ZjQ8ckblAQBAl2YW0KvqhUl+LcnfZQjnn1l/jQPy6NHwYzMsEwAAujOTgF5Vv5DhotA9SZ7QWvvsOsseXlUnVdXxK6Y/pKqOXmX5B2TYK58kb5pFfQEAoFdTXyRaVc9I8tIkt2fogeW5VbVysb2ttYtG48ckuSrJdUmOHVvmqUnOqapLk1yboReX45N8T5I7J3l7kldOW18AAOjZLHpxeeBoeMckP7XGMu9KctEG5Vya5EFJvi3DKS1HJvlCkssy9It+cdvMuyoBAEAHpg7orbVzM3R/OOnye5N8zS720U2I3IgIAIAdbebdLAIAAAdPQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoyNQBvaruWVU/VlVvqaqPVtWtVXVTVV1WVT9aVQf0GlV136p6Q1V9sqpuq6q9VXV+Vd1j2roCAEDvDptBGU9N8vokNyS5NMnHk9w7yfcn+c0k311VT22ttY0Kqqrjk1yR5F5J3prkQ0m+I8nzkjyxqk5trX1uBnUGAIAuzSKgfyTJk5Nc0lrbtzSxql6U5K+T/ECGsP4HE5T1ugzh/LmttVePlfWrSZ6f5GVJnj2DOgMAQJemPsWltfYXrbU/Hg/no+mfSvLro6e7Niqnqo5LcnqSvUleu2L2S5LckuTpVXXktHUGAIBezWIP+nq+PBp+ZYJlHz8avnOVsL9YVZdnCPCPSvLn6xVUVXvWmHXS4uJidu/ePUF1dpbFxcUk0TZT0IbT04az0UM7nv3QSf7tH7zNfm89tOF2pw2npw2nN882XHrtg7FpAb2qDkvyI6On75hglQeNhh9ZY/7VGQL6idkgoLN9XXn9TZta/snH3G1Ty//Sl/flum3+HtjYdt9OgWU+z/RoM/egvzzJyUne3lr70wmWX9qC1/qkLE2/+0YFtdZOWW16Ve1ZWFh4+K5duyaozs6y9Mty3m3zzHMu2dTy9z5t16aVvXv37lx3/U155Qc398DUZr6HeetlO9xI79tpD+3YexttpIc23O62Sxv2vK1ulzbs2TzbcGFh4aDX3ZR+0KvquUlekKEXlqfPqtjRcMPeYAAAYLuaeUCvqrOSvCrJPyY5rbV244SrLu0hX+tY0FErlgMAgEPOTAN6Vf1UktckuTJDOP/UAaz+4dHwxDXmnzAarnWOOgAAbHszC+hV9cIkv5bk7zKE888cYBGXjoanr7z7aFUtJDk1ya1J3jtlVQEAoFszCehV9QsZLgrdk+QJrbXPrrPs4VV10uiuoV/VWrsmyTuTHJvkrBWrnZfkyCRvbK3dMos6AwBAj6bubqKqnpHkpUluT/KeJM+tqpWL7W2tXTQaPybJVUmuyxDGxz0nyRVJLqiqJ4yWe2SS0zKc2vLiaesLAAA9m0V/cA8cDe+Y5KfWWOZdSS7aqKDW2jVV9YgMgf+JSZ6U5IYkFyQ57wAuOAUAgG1p6oDeWjs3ybkHsPzeLHeZuNr8TyR51rT1AgCA7WhT+kEHAAAOjoAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0JGZBPSq+sGqenVVvaeqvlhVraredBDl7B2tu9rjU7OoKwAA9OywGZXz80keluTmJP+U5KQpyropyfmrTL95ijIBAGBbmFVAf36GYP7RJI9LcukUZX2htXbuLCoFAADbzUwCemvtq4G8qmZRJAAA7Eiz2oM+S0dU1Q8nuX+SW5J8IMm7W2u3z7daAACw+aq1NtsCq3ZlOMXld1trP3yA6+5N8oBVZl2b5FmttXdNWM6eNWaddMIJJ9z1wgsvPJBq7QiLi4tJkoWFhbnW48rrb9rU8k8+5m6bVvbi4mK+9OV9+fStm/YSSTb3PcxbL9vhRnrfTntox97baCM9tOF2t13asOdtdbu0Yc/m2YZnnnlmrr766ve11k450HV762bxt5M8Icl9khyZ5KFJfiPJsUn+pKoeNr+qAQDA5uvqFJfW2nkrJl2Z5NlVdXOSFyQ5N8lTJihn1V8qVbVnYWHh4bt27Zqypoee3bt3J0nm3TbPPOeSTS1/79N2bVrZu3fvznXX35RXfnBzP1ab+R7mrZftcCO9b6c9tGPvbbSRHtpwu9subdjztrpd2rBn82zDafba97YHfS2/Pho+dq61AACATbZdAvpnRsMj51oLAADYZNsloD96NPzYXGsBAACbbMsDelUdXlUnVdXxK6Y/pKqOXmX5ByR5zejpm7aijgAAMC8zuZqtqs5Icsbo6X1Gw0dX1UWj8c+21s4ejR+T5Kok12XonWXJU5OcU1WXZuhWcTHJ8Um+J8mdk7w9yStnUV8AAOjVrLqb+NYkz1gx7bjRIxnC+NlZ36VJHpTk2zKc0nJkki8kuSzJxUkubrPutB0AADozk4DeWjs3QxeIkyy7N0mtMv1dSSa6EREAAByqtstFogAAsCMI6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdmUlAr6ofrKpXV9V7quqLVdWq6k0HWdZ9q+oNVfXJqrqtqvZW1flVdY9Z1BUAAHp22IzK+fkkD0tyc5J/SnLSwRRSVccnuSLJvZK8NcmHknxHkucleWJVndpa+9xMagwAAB2a1Skuz09yYpKjkvzEFOW8LkM4f25r7YzW2jmttccn+bUkD0rysqlrCgAAHZtJQG+tXdpau7q11g62jKo6LsnpSfYmee2K2S9JckuSp1fVkQddUQAA6FxPF4k+fjR8Z2tt3/iM1tpiksuT3DXJo7a6YgAAsFVqip3eqxdYtSvJpUl+t7X2wwew3iuSnJ3k7Nbar6wy/zVJzkrynNba6zcoa88as0464YQT7nrhhRdOWq0dY3FxMUmysLAw13pcef1Nm1r+ycfcbdPKXlxczJe+vC+fvnXTXiLJ5r6HeetlO9xI79tpD+3YexttpIc23O62Sxv2vK1ulzbs2Tzb8Mwzz8zVV1/9vtbaKQe67qwuEp2FpS14rU/K0vS7b35VNsdm/xOYxr3vMgyvW6eOh3Iw3E56/jKZxHr1n2Q7nMR231an/Rv7PM/Gl768b+ptcT2Hwt9gsz/Ph0IbHQp6zi8bufddkjsf3tMJI5PpKaBvpEbDDXf5r/VLpar2LCwsPHzXrl2zrNfEnnnOJXN53Umc/dCvJEle+cG1N4m9T9u16fXY7DbazPewe/fuXHf9Teu24Xaw2X/n9f7Gk2yHk5jne+hBD5/n7fxZTrbm87wV/1M322Z/nnf6987u3buTJPPKLUt6/5+3nrMf+pWcfPTCXNpwmr32Pf2kWPp5ttbP5aNWLAcAAIecngL6h0fDE9eYf8Jo+JEtqAsAAMxFTwH90tHw9Krar15VtZDk1CS3JnnvVlcMAAC2ypYH9Ko6vKpOGt019Ktaa9ckeWeSYzP01jLuvCRHJnlja+2WLakoAADMwUyufqmqM5KcMXp6n9Hw0VV10Wj8s621s0fjxyS5Ksl1GcL4uOckuSLJBVX1hNFyj0xyWoZTW148i/oCAECvZnV5+rcmecaKaceNHskQxs/OBlpr11TVI5K8NMkTkzwpyQ1JLkhyXmvtxhnVFwAAujSTgN5aOzfJuRMuuzfLXSauNv8TSZ41i3oBAMB209NFogAAsOMJ6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdmVlAr6r7VtUbquqTVXVbVe2tqvOr6h4HUMbeqmprPD41q7oCAECvDptFIVV1fJIrktwryVuTfCjJdyR5XpInVtWprbXPTVjcTUnOX2X6zTOoKgAAdG0mAT3J6zKE8+e21l69NLGqfjXJ85O8LMmzJyzrC621c2dULwAA2FamPsWlqo5LcnqSvUleu2L2S5LckuTpVXXktK8FAACHulnsQX/8aPjO1tq+8RmttcWqujxDgH9Ukj+foLwjquqHk9w/Q7j/QJJ3t9Zun0FdAQCga9Vam66AqlckOTvJ2a21X1ll/muSnJXkOa21129Q1t4kD1hl1rVJntVae9eEddqzxqyTTjjhhLteeOGFkxQzc1def9NcXncS977LMPz0rWsvc/Ixd9v0emx2G23me1hcXMyXvrxv3TbcDjb777ze33iS7XAS83wPPejh87ydP8vJ1nyet+J/6mbb7M/zTv/eWVxcTJIsLCzMqjoHpff/eeu5912SOx9+h7m04Zlnnpmrr776fa21Uw503Vn04rK05a3111uafvcJyvrtJE9Icp8kRyZ5aJLfSHJskj+pqocddC0BAGAbmNVFouup0XDDXfWttfNWTLoyybOr6uYkL0hybpKnTFDOqr9UqmrPwsLCw3ft2rVREZvimedcMpfXncTZD/1KkuSVH1x7k9j7tF2bXo/NbqPNfA+7d+/OddfftG4bbgeb/Xde7288yXY4iXm+hx708Hnezp/lZGs+z1vxP3Wzbfbnead/7+zevTtJMq/csqT3/3nrOfuhX8nJRy/MpQ2n2Ws/iz3oS3vI1zqGc9SK5Q7Gr4+Gj52iDAAA6N4sAvqHR8MT15h/wmj4kSle4zOjoZ5gAAA4pM0ioF86Gp5eVfuVV1ULSU5NcmuS907xGo8eDT82RRkAANC9qQN6a+2aJO/McCHnWStmn5dhr/cbW2u3JElVHV5VJ43uPvpVVfWQqjp6ZflV9YAkrxk9fdO09QUAgJ7N6uqX5yS5IskFVfWEJFcleWSS0zKc2vLisWWPGc2/LkOoX/LUJOdU1aUZulVcTHJ8ku9Jcuckb0/yyhnVFwAAujSTgN5au6aqHpHkpUmemORJSW5IckGS81prN05QzKVJHpTk2zKc0nJkki8kuSzJxUkubtN22g4AAJ2bWf9RrbVPJHnWBMvtzXLXi+PT35VkohsRAQDAoWoWF4kCAAAzIqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdGRmAb2q7ltVb6iqT1bVbVW1t6rOr6p7zKMcAADYjg6bRSFVdXySK5LcK8lbk3woyXckeV6SJ1bVqa21z21VOQAAsF3Nag/66zKE6ue21s5orZ3TWnt8kl9L8qAkL9vicgAAYFuaOqBX1XFJTk+yN8lrV8x+SZJbkjy9qo7cinIAAGA7m8Ue9MePhu9sre0bn9FaW0xyeZK7JnnUFpUDAADbVrXWpiug6hVJzk5ydmvtV1aZ/5okZyV5Tmvt9ZtdzmjZPWvMetgRRxxxx/vf//7rrb5pvvTl2+fyupM4bPRT7Sv71l7mzoffcdPrsdlttJnvYd++fdnX2rptuB1s9t95vb/xJNvhJOb5HnrQw+d5O3+Wk635PG/F/9TNttmf553+vbNv39B4d7jDfDvd6/1/3noOu0Nyh6q5tOHHP/7x3HbbbTe21u55oOvO4iLRu42GN60xf2n63beonPXcftttt9109dVX752ijEPVSaPhh+Zai+1NG05PG86GdpyeNpyeNpyeNpzePNvw2CRfPJgVZ9KLywZqNJxuV/0BlNNaO2XK19pxlo46aLuDpw2npw1nQztOTxtOTxtOTxtOb7u24Sz29y/t2b7bGvOPWrHcZpcDAADb1iwC+odHwxPXmH/CaPiRLSoHAAC2rVkE9EtHw9Orar/yqmohyalJbk3y3i0qBwAAtq2pA3pr7Zok78xwIvxZK2afl+TIJG9srd2SJFV1eFWdNLpr6EGXAwAAh6JZXST6nCRXJLmgqp6Q5Kokj0xyWoZTUl48tuwxo/nXZQjjB1sOAAAccqbuB/2rBVXdL8lLkzwxyT2T3JDkj5Kc11q7cWy5Y5Ncm+S61tqxB1sOAAAcimYW0AEAgOnN99ZUAADAfgR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI7M6kZFHAKq6luS/G2Sw8cmP7C1tnc+NdoequroJP8hyZOTPCjJvZP8a5JPJbkmye4k72itfXBedexVVd09ydOSPCnJt2S498FhSW7KcHOyyzPcQfjKedVxXqrq65OcMvZ4RJL7jy1yWmtt9wGWeUSSH0ny1CTfnOQbktyY5Ookf5jkDa21L05d+U7Msg1H9+g4PcmuDNvqAzLc4XoxyceTvDfJxa21y2dU/S5sxna4ymvcIcllSR49Nvm81tq505Tbi81qw6qqDPeM+YEk/zbJfZLcJclnknwyw40fL03yp621L0/xFrqwSf8TvyvD9/e3J7lvkq9LckuG9ntfkj9I8tbW2u3T1v+AtdY8PJIhFO1J0lY8jp133Xp+JDkzyedWabeVj93zrmtvjyRPSfLPE7TdviS/nuTwedd5C9vm2RO0y64DLPPkJFduUOZ1SR4z7/ffUxsmOSpD0Nk3QXktySVJ7jPv999TG07wOi9Ypdxz5/3+e27DJA/NsANjkm3y2Hm3Q2/tmOGu9u+asP0+kOSbt/o924POkp9L8vAMe37vNOe6bAtV9coMXyxLPpThA399hh88/ybJ8UlO3fra9a2qvjfJ72f5NLt/SfK2JB9LcluGvSLfnWGPUCX5T0nunmFPx05w51WmLY6mH77KvHWN7uD8ZxnaM0m+kGHP0N4M2+n3j+bdP8nbq+pxrbX3H+jrdGZWbXjX7L9n96YMgf3KJJ9Pco8Me9W/fTT/SUl2V9VjWmufPcA692am2+FqqurEJL84enoofv/MvA2r6tQkb8/w4zEZPs9/nuGo480ZjuIek+G75z6rFLEdzawdq2ohw3f18WOTL0/y/iSfztB+p2T5c//QJO+uqoe11q4/wHofNAGdVNXJSX5+9PRlSc6bY3W2hao6O8vh/IYkP9Zae/sayx6Z5KStqlvvRoezX5vlcP4/k/zoyjBTVXfOsF2+eDTp31fVa1prl21ZZefn5gxfIHvGHh9Jcm2GUysO1G9l+Yv68iTf11r73NLM0fb8e0nOSLKQ5E1V9S1tHod1Z2fWbfiOJL+R5JK2yukCox+dv5fhEPmDkvxKkmccVM37Mes23M/of8EbMpyW8aEkf5dD70f4TNuwqh6Y5E8yfE5bkpcn+aXW2s2rLFtJvi3DkcrtbpbteE6Ww/k/J3lKW+XUtKralWFH0j1Hj/OS/NiBV/0gzfuwhcd8Hxl+pP1thg/6BzP8Ej2kDo1tQpudlOTWUfvcmOE8/bnXa7s8kjxqbPtaTHLUBstfNrb8y+Zd/zm33d4c4OHcJP/H2DqfS3LvNZa7a4Zz0ZeW/b/n/X57aMMM55l/+4Rl//uxsr+c5B7zfr89tOE65Tw/y6exnZrkorFyz533++yxDTOcU7603vPm/T7m/TjI/4kfGlvnWRss+2Njy16/le9NLy68MMOhnH0Z9mJu+wtJtsDPZflw28+21q6dZ2W2oWPHxq9sG1+U+J6x8a+ffXUOeT8yNv4brbVPr7ZQa+1fkrxyjfV2rNbaLa21v5lw8TdnuDg8GXZ+fPs6y+5oVfVNSf7L6Olr2yF2ce1mqKrTMpxOlSSXt9ZeNcfqbGfHjo1fscGyc/v+EdB3sKp6SJL/Z/T0Va21v55nfbaDUY8tPzR6emOSi+dYne3q1rHxSf7hfcPY+HUzrsshbXQKwfeOTXrzBqv8foY9RUnynaNeE5hQG3a57R2bdM85VaVro1Mv3pDhqM3HM+z0YGPPHhs/f16VOAQcyHfQ3L5/BPQdqqrumOS3M1yQc22Wz0FnfbuyvPf8L1prt1XVt1bVhVV1TVV9qapurKr3V9UrRhfnsb/3JvnKaPybqurJay1YVcdl6BIwGU4Z+P82uW6HmuOS3G00vpihN4I1teG89KtGT++QoTtBDswxY+OHwrm/m+Enk3znaPw/tVXOn2Z/ox813zV6enuSP6uqo6rqhVX1N1X1+ar6l6q6tqp+r6q+Z47V7d34XvGfHrXt1xjt4Dh7bNIbN7VWKwjoO9fPZPnw65mjw9ts7FFj4++vqhcl+ZskP54hDB2RoVeHb83wwf5wVb1gZSE72egUi9ePTXpzVV1QVadU1dFVdWRVPbiqzskQ5o/KEM5/vLX2sXnUeRt7yNj4Na21fROs89E11mcDVfWoJPcbPf3XJI5KrjD60f1Lo6dvaq29Y5712UYenOUf2x9LcmKG68ZenqE/8LtnuNj22CT/McnbqurPquoeW17T/r0sw4+cZOjB6l1V9eSqul9VHVFV96+qMzJc//R9o+XeluS/bWUl9eKyA1XVg5OcO3r6htba/5pjdbabE8bGvzfLgf3jGXojuSFDbxnfm+Ef5Z2SvLKq7tRa+69bWM/e/XSSOyZ5ToYfNT85eqz0lQzdiZ3nFKyDMt7F2j9NuM4n1lifdYz2wr1ibNJ/n+D6ih1l1Ea/leHC239O8lNzrdD2Mv7dUxn+L359hm5p/2eGsH6nDDcsevxouX+X5NKqenRrbfy0jh2ttfZXo16X/nuGHz3fmeUjOitdleE+HK8encK2ZexB32HGTm05IsPFTGevvwYrjO+NWArnFyY5sbX2k621X2qtPTdDN2uvG1v2F6vqYVtVyd611r7SWjsrQ88N6+1B+3iGbgE/siUVO/QsjI3fMuE640fTFtZcipVelOQxo/EvJvmFOdalV2dl+SLH57axrj7Z0Ph3zzdlCOcfTfLQ1toPtdZ+sbX2C621J2S4u+jS5/1hSV66tVXtX2vtTzL0yHZehh85q7ktyV8lec9Wh/NEQN+JXpDkkaPxs1prn59nZbahr1vx/PIkz26t7fcBb639a5L/nKHf1mTYW/wzm1+97WN0OsDLMpxXeUuGCxjPzXA9xG9muNXycaNl/qGqHrl6SazjLmPj/zrhOl8aG7/rDOtyyKqq78/yzXaS4bTBT6y1/E406r/75aOnb2utuZ7kwKz87vlykjNaa1evXLC19qfZ/4jkT1TV3TexbtvOqD1elGEn5RFJ/jLDEbAXjYZ/OZr+zCR/W1Uv2eo6OsVlB6mqB2X5JkR/2Fr7w3nWZ5taeZjwl9f6Zd1aa1X1X5M8bjTpSVVV8/gl3puqenqGIzl3zHCHy/+rfe2Nig7PEM5/Jsk3JnnH6OY5gs/kxrfXSe/QOH7HPtembKCq/l2GGxQtXWj2stba/5hjlbqz4tSWLyb5ifnWaFta+d3zx621f1hn+Tdm+NF4TIZ235XkjzalZttMVd0vye4MO4BuTPLk1tpfrLLc45P8/0mOTnJuVd3YWnv1VtXTHvQdYnQ18m9n+PL9QoZDjRy4xbHxluFDvp53Z7nHknskeeAm1GlbGf1Q/H8zhPNrM+wF+ppborfWvtxa+9kM/yCT4SIoh2oPzPj2euSE64zvNV9ccymW7jT41gx72pLkV1presT6Ws9Octpo/Gdba5NeD8GylZ/FrwmU49pwF+B3jU16+MxrtH1dnCGcJ8kPrxbOk2Q0/eljk/5LVW3ZaX8C+s7xrCSPHo2/oLX2qfUWZk3j7XZTa23dADO6MOfGsUn6lR4OvS4Fmt+YoAehXx0bP2P0Y5PJjG+vx6y51P7uOza+6k2NSKrqcRl6dlj6QXN+a801PSuMTiVY6v3i3Rmu2eHArfzOnuRHzvgyvnuSVNUjsnxU+0Ojc9HX1Fp7e5IPj54eleHC2y3hi27nuN/Y+G9VVVvrsWK9a8fmnbuF9e3VlQexTltjfKd69Nj4BydYfrzv7rvHF82B+Mex8eMn/HHzTWPj6x1C37FG4fySLB+VuKC19vw5Vqlnd8/y+dOPTbJvne+eZ4yt95KxeXu3uM498t0zGwf6/ZPs/x10wppLzZiADgdmz9j43apq5YU7+6mqO2f/uwm6ccn+PYMczJfGJH15M7gmyU2j8aOSPHS9hUd3yv3m0dN9Sf5+86q2Pa0Szl/TWnveHKvEDtBauzH736V2kiNi40fDfPcMDub7Z/xGRlv2/eMi0Z3j75L8zoTLju/F+IMkS3d5+7sZ1me7uizDP7pvyPCh3ZXhMPdaHpvlz9lnWmt7N7Ny28Rns7wX4luSrHuIMUM3YUtuy/6nDLGO1tq+qnpbkqeNJj0164fuH8jyl9Flq10bsJONzjm/JMuntbyutbZa//0suzmTf/c8Jsnxo/G/z/J3ju1w8JYkS0dqHp/9u/Ldz6hL5ceNTXIficH4tjTpnZLHl7thhnVZl4C+Q7TW/igTXsFdVeMB/Wyhcllr7faqujjDjXaS5IVVdclqPbOMei74ubFJb9mKOm4Df5Xlw4xnVtUFG9xE46fHxi+b8G6YLLs4ywH92aP2/szKharqLtm/K9Atva1179YI5y6238DoR94zJ1m2qi7KckD/o9bauZtTq23rdzLc3KmSPLmqvrm19o9rLPsjWd6DfmM27tBgp/irsfGTqur01to711q4qv7PDHdtXfLuTavZCk5xgQP3S0mW+o9/TJLXV9UR4wuMugh8dZZvynFrkl/eqgp27qIsHyY8LskfVtU9Vy5UVXeqqlcm+cGxyWvuMWJ1oz6Rd4+e3jPJH4xOZfmqUTh/U5aPbHwok+/1POStEs5fn+E+B7BlWmt/n+FzmiSHJ/mjqvqmlctV1XcluWBs0ismuBh/Rxi14fipqr87Om3ta4y6UB3fUfHHW9nNb+mSmZVWXCj6QHvQv9boV/VbsnwU6uMZulu7IcPt0Z+c5NjRvJbkGa21i7e4mt2qqpcneeHYpJsznCp0VYZuKY9N8j0Z+j9f8j9aa/9hq+o4b1X1+6tM/u4sh8R352vPK31za+3Nq5R1XJIrktx7NOnzSX4/yXUZttcfSPJvRvNuTrKrtbZnZTnbzSzasKpOyHCqxdI6n0xyfiY7f/XK1tp6d8rt3iy3wwle66Isn2J53qGyB33Gn+Wjk7wny9eK3Jbhu+fKDKH93yZ5wtgqf5bku0fdLm5rs2rHUU8uu7N/17OXjx6fz9Al8ndm/wtKP53k0a21aw+2/gestebhsd8jwxfP0uPYeden10eS70vymRXttfJxU5Ifmndde3xkuGPbbRu0X8uwt/1VSe407zpvcfts1C6rPc5dp7xvydCry3rrfyLJY+f93ntqwwxHwQ6mnJbkonm3QQ9teACvddG0ZfT42ITP8jdm6Ad9ozJ+J8md5/3+e2zHDOH76gnLeF+Sk7b6/ToHHQ5Sa+2tVXVZhj7mz8jQPd3RGUL5RzJc/Pj61trn5lbJjrXWfqmq3phhj9lpGfYI3SPDDYy+kOGf53uSvKG19pF51fNQ0Vr7QFV9W4b2fmqG9v76DHuMPprkD5P8Zmvti/OrJbCR1tonkzy+qs5I8h+TfHuGI2G3J7k+w57k32qt/dWahexwrbW/rKqTkzwlw862h2dowyOT3JLhSNnfZDjS+LY2h2ufnOICAAAdcZEoAAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADryvwE3THmyhrM3HAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 254,
       "width": 372
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "counts.hist(bins=25,xlabelsize=16);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9.8"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counts.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To standardize to zero mean and unit standard deviation, we can use tools from the `scikit-learn` library.\n",
    "\n",
    "(We will discuss scikit-learn more in upcoming lectures.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvIAAAIbCAYAAACJ0HoGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAABYlAAAWJQFJUiTwAAAvIElEQVR4nO3df5RsZ1kn+u8TkoHk2AkkSqJBiMkEggQRwhIY/HFCxgygIoqZe70RDTOayyVehEVcRtBFwlxGZkCBACIZB1gEvHccGQY1qHEwJxAi6hxQJhIgRE74GREC4SSEKMl7/6hqTqXTfbpO166ufrs/n7V67apdb737raffU+dbu3ftXa21AAAAfTls0QMAAAAOnSAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHoBVVdUJVfXqqrqxqr5WVX9fVX9QVWctemwAJNVaW/QYANhiquq7kvxZkuPGq76S5Jsy2gHUkrywtfayBQ0PgNgjD8AKVXVkkt/PKMR/MMnprbVjkjwgya8nqSS/VlVnL26UANgjD8A9VNXzkrwyyW1JTmutfWbF4+9I8vQkH2itnbHpAwQgiT3yANzbuePl76wM8WMvHy8fU1WnbdKYAFhBkAfgG6pqKcnyXvY/WaPZ+5PcOr79pLkPCoBVCfIATHp4RsfAJ8nfrtagtXZ3ko+O737nZgwKgHsT5AGY9K0Ttz97kHbLj33rQdoAMEeCPACTdk3cvuMg7b46Xn7THMcCwEEI8gBMqvWbALAVCPIATLpt4vaRB2l31CrtAdhEgjwAkyaPi/+2g7RbfuxzcxwLAAchyAMw6SNJlq8U+IjVGlTVYUkeNr774c0YFAD3JsgD8A2ttf1J/uf47g+u0exxSY4Z33733AcFwKoEeQBW+p3x8tyqWu30kheOl3tbax9d5XEANoEgD8BKb0hyU5KlJH9YVd+ZjK76WlX/McmPj9u9cEHjAyBJtdbWbwXAjlJVj8rosJnjxqu+ktE54w/L6Bj6F7bWXrag4QEQQR6ANVTVCUl+OckPJzkxozD/l0le2VpzbDzAggnyAADQIcfIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADo0OGLHsBmqqpPJDk6yb4FDwUAgO3tpCRfaa19x7w2sKOCfJKjjzzyyGMf/vCHH7vogRyK/fv3J0mWlpYWPJLtR23nR23nR23nQ13nR23nR23nZ9baXn/99bnjjjuGHNK97LQgv+/hD3/4sXv37l30OA7Jnj17kiS7d+9e6Di2I7WdH7WdH7WdD3WdH7WdH7Wdn1lre8YZZ+QDH/jAvsEGtArHyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0aJAgX1X/oareXVWfqqo7quqWqvpgVb24qo47xL4eVFVvrKrPVtWdVbWvql5VVQ8YYqwAALAdDLVH/vlJdiX50ySvTvK2JF9PcnGSD1XVt0/TSVWdkmRvkmcl+cskr0zyd0l+IcmfH+qHAgAA2K4OH6ifo1trX1u5sqpemuSFSX45yXOm6Oc3kzwwyXNba6+Z6Oc3Mvqw8NIkzx5kxAAA0LFB9sivFuLHfne8PHW9Pqrq5CRnJ9mX5HUrHn5xktuTPLOqdm1wmAAAsG0MtUd+LT8yXn5oirZPGi+vbK3dPflAa21/Vb0vo6D/+CTvPlhHVbV3jYdO279/f/bs2TPFcLaO/fv3J0l34+6B2s6P2s6P2s6Hus6P2s6P2s7PrLVdfv48DRrkq+rCJN+U5Jgkj03yvRmF+JdN8fSHjZcfW+PxGzIK8g/NOkEegMW67jO3zrX/0088Zq79A/Rg6D3yFyY5fuL+Hyc5r7X2D1M8d/ldea13/+X191+vo9baGautr6q9S0tLj9m9e/cUw9k6lj8J9jbuHqjt/Kjt/PRQ2/MuumKu/e87d/fgffZQ116p7fyo7fzMWtulpaXhBrOGQc8j31o7obVWSU5I8uNJTk7ywap6zADd1/JmBugLAAC6NpcLQrXW/r619o6MDoU5Lslbpnja8h73tf5eevSKdgAAsGPN9cqurbWbknw4ySOq6pvXaf7R8fKhazy+fOabtY6hBwCAHWOuQX7s28bLu9Zpd9V4eXZV3WNcVbWU5IlJ7kjy/mGHBwAA/Zk5yFfVaVV1wirrDxtfEOqBSa5trX1pvP6I8XNOmWzfWrsxyZVJTkpywYruLsnoyrFvaa3dPuuYAQCgd0OctebJSV5eVe9JcmOSL2Z05pofyOjLrjcn+bmJ9icmuT7JTRmF9knPSXJtkkur6qxxu8clOTOjQ2peNMB4AQCge0ME+f+R5LKMDn15VEanh7w9o+B9eZJLW2u3TNNRa+3Gqnpskpdk9AHhqUk+l+TSJJdM2w8AAGx3Mwf51tp1ufehMAdrvy8HTiW52uOfSvKsWccFAADb2WZ82RUAABiYIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQoZmDfFUdV1U/W1XvqKqPV9UdVXVrVV1TVf+2qqbeRlXtq6q2xs/Ns44VAAC2i8MH6OOcJK9P8rkkVyX5ZJLjk/x4kt9O8pSqOqe11qbs79Ykr1pl/W2zDxUAALaHIYL8x5I8LckVrbW7l1dW1QuT/GWSZ2QU6t8+ZX9fbq1dPMC4AABg25r50JrW2p+11v5gMsSP19+c5LfGd3fPuh0AAOCAmv6Ilw10XvWLSf5jkle11p4/Rft9Se6b5BeTPDjJ7Uk+lOQ9rbW7DmG7e9d46LRTTz31qMsuu2zarraE/fv3J0mWlpYWPJLtR23nR23np4faXveZW+fa/+knHjN4nz3UtVdqOz9qOz+z1vb888/PDTfc8IHW2hlDjmvSEIfWrKqqDk/y0+O7f3wITz0hyeUr1n2iqp7VWrt6kMEBAEDn5hbkk7wsyelJ3tVa+5Mpn/OmJO9N8rdJ9ic5OcnPJzk/yR9V1RNaa3+zXidrffKpqr1LS0uP2b1795TD2Rr27NmTJOlt3D1Q2/lR2/npobbnXXTFXPvfd+7uwfvsoa69Utv5Udv5mbW2m/FXkrkE+ap6bpIXJPlIkmdO+7zW2iUrVl2X5NlVddu4v4uT/NhAwwQAgG4NfkGoqrogyauTfDjJma21WwbodvlLs98/QF8AANC9QYN8VT0vyWsz2pN+5vjMNUP4/Hi5a6D+AACga4MF+ar6pSSvTPLXGYX4zx/8GYfkCePl3w3YJwAAdGuQIF9Vv5rRl1v3JjmrtfaFg7Q9oqpOq6pTVqx/RFUdu0r7h2S0lz9J3jrEeAEAoHczf9m1qn4myUuS3JXRGWeeW1Urm+1rrb15fPvEJNcnuSnJSRNtzklyUVVdleQTGZ215pQkP5TkfkneleQVs44XAAC2gyHOWvMd4+V9kjxvjTZXJ3nzOv1cleRhSR6d0aE0u5J8Ock1GZ1X/vI2z6tXAQBAR2YO8q21izM6LeS07fcludcu+/HFnlzwCQAApjD46ScBAID5E+QBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOiTIAwBAhwR5AADokCAPAAAdEuQBAKBDgjwAAHRIkAcAgA4J8gAA0CFBHgAAOjRzkK+q46rqZ6vqHVX18aq6o6puraprqurfVtUhbaOqHlRVb6yqz1bVnVW1r6peVVUPmHWsAACwXRw+QB/nJHl9ks8luSrJJ5Mcn+THk/x2kqdU1TmttbZeR1V1SpJrkzwwyTuTfCTJ9yT5hSRPrqontta+OMCYAQCga0ME+Y8leVqSK1prdy+vrKoXJvnLJM/IKNS/fYq+fjOjEP/c1tprJvr6jSTPT/LSJM8eYMwAANC1mQ+taa39WWvtDyZD/Hj9zUl+a3x393r9VNXJSc5Osi/J61Y8/OIktyd5ZlXtmnXMAADQuyH2yB/MP42XX5+i7ZPGyytX+VCwv6rel1HQf3ySdx+so6rau8ZDp+3fvz979uyZYjhbx/79+5Oku3H3QG3nR23np4faXvjIad72N24er72HuvZKbedHbedn1touP3+e5hbkq+rwJD89vvvHUzzlYePlx9Z4/IaMgvxDs06Qp1/XfebWufZ/+onHzLX/eY8/mf9rYH29z1PgAP+e6dk898i/LMnpSd7VWvuTKdovz/S1/kUtr7//eh211s5YbX1V7V1aWnrM7t27pxjO1rH8SbC3cW/EeRddMdf+9527+x73h67tvMef3Ps1bFXbed5u9jxdqYfaLrpGG9FDXXu1lWvb41ydtJVr27tZa7u0tDTcYNYwl/PIV9Vzk7wgo7POPHOobsfLdc9+AwAA293gQb6qLkjy6iQfTnJma+2WKZ+6vMd9rb9BHb2iHQAA7FiDBvmqel6S1ya5LqMQf/MhPP2j4+VD13j81PFyrWPoAQBgxxgsyFfVLyV5ZZK/zijEf/4Qu7hqvDx75dVgq2opyROT3JHk/TMOFQAAujdIkK+qX83oy617k5zVWvvCQdoeUVWnja/i+g2ttRuTXJnkpCQXrHjaJUl2JXlLa+32IcYMAAA9m/msNVX1M0lekuSuJO9N8tyqWtlsX2vtzePbJya5PslNGYX2Sc9Jcm2SS6vqrHG7xyU5M6NDal4063gBAGA7GOL0k98xXt4nyfPWaHN1kjev11Fr7caqemxGHwyenOSpST6X5NIklxzCF2cBAGBbmznIt9YuTnLxIbTflwOnklzt8U8ledas4wIAgO1sLueRBwAA5kuQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOjQIEG+qn6iql5TVe+tqq9UVauqt26gn33j5672c/MQYwUAgO3g8IH6+ZUkj0pyW5JPJzlthr5uTfKqVdbfNkOfAACwrQwV5J+fUYD/eJIfSHLVDH19ubV28RCDAgCA7WqQIN9a+0Zwr6ohugQAAA5iqD3yQ7pvVf1UkgcnuT3Jh5K8p7V212KHBQAAW0e11obtsGp3RofWvK219lOH+Nx9SR6yykOfSPKs1trVU/azd42HTjv11FOPuuyyyw5lWAu3f//+JMnS0tKCRzJ/133m1rn2f/qJx9zj/tC1nff4k3u/hq1qO8/bzZ6nK/VQ20XXaCN6qGuvtnJte5yrk7ZybXs3a23PP//83HDDDR9orZ0x5LgmbbXTT74pyVlJTkiyK8kjk7whyUlJ/qiqHrW4oQEAwNaxpQ6taa1dsmLVdUmeXVW3JXlBkouT/NgU/az6yaeq9i4tLT1m9+7dM450c+3ZsydJ0tu4N+K8i66Ya//7zt19j/tD13be40/u/Rq2qu08bzd7nq7UQ20XXaON6KGuvdrKte1xrk7ayrXt3ay13Yy/kmy1PfJr+a3x8vsXOgoAANgiegnynx8vdy10FAAAsEX0EuSfMF7+3UJHAQAAW8SmB/mqOqKqTquqU1asf0RVHbtK+4ckee347ls3Y4wAALDVDfJl16p6epKnj++eMF4+oarePL79hdbahePbJya5PslNGZ2NZtk5SS6qqqsyOt3k/iSnJPmhJPdL8q4krxhivAAA0Luhzlrz3Ul+ZsW6k8c/ySi0X5iDuyrJw5I8OqNDaXYl+XKSa5JcnuTyNvRJ7wEAoFODBPnW2sUZnRpymrb7ktQq669OMtUFnwAAYKfr5cuuAADABEEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEODBPmq+omqek1VvbeqvlJVrareusG+HlRVb6yqz1bVnVW1r6peVVUPGGKsAACwHRw+UD+/kuRRSW5L8ukkp22kk6o6Jcm1SR6Y5J1JPpLke5L8QpInV9UTW2tfHGTEAADQsaEOrXl+kocmOTrJ/zVDP7+ZUYh/bmvt6a21i1prT0ryyiQPS/LSmUcKAADbwCBBvrV2VWvthtZa22gfVXVykrOT7EvyuhUPvzjJ7UmeWVW7NjxQAADYJrbSl12fNF5e2Vq7e/KB1tr+JO9LclSSx2/2wAAAYKupGXair95h1e4kVyV5W2vtpw7heS9PcmGSC1trv77K469NckGS57TWXr9OX3vXeOi0U0899ajLLrts2mFtCfv370+SLC0tLXgk83fdZ26da/+nn3jMPe4PXdt5jz+592vYqrbzvN3sebpSD7VddI02ooe69mor17bHuTppK9e2d7PW9vzzz88NN9zwgdbaGUOOa9JQX3YdwvJMX+tf1PL6+89/KPOx0TeL448cLW/ahJB4ML0EyO2u9/90fNhZ33o1GuI9ofcabYbe/61thmlrtNE5ux1qtB1sxvv2PG3nebSVgvx6arxc908Ia33yqaq9S0tLj9m9e/eQ45raeRddsaHnXfjIrydJXvG/Fvvr2nfu7rlvY6M1mtbK17Bnz54kyVBzYt7j3wxD/Z7Xqu1m1Gjec3XRv+ch3hN6r9E8xr9yzvb4GjbbtDXa6Jzdjv/vDG3o/8dWs+j3vFlt9Hcwa203468kW+kY+eWPe2t9bDp6RTsAANixtlKQ/+h4+dA1Hj91vPzYJowFAAC2tK0U5K8aL8+uqnuMq6qWkjwxyR1J3r/ZAwMAgK1m04N8VR1RVaeNr+L6Da21G5NcmeSkjM5OM+mSJLuSvKW1dvumDBQAALawQb49WVVPT/L08d0TxssnVNWbx7e/0Fq7cHz7xCTXJ7kpo9A+6TlJrk1yaVWdNW73uCRnZnRIzYuGGC8AAPRuqNOgfHeSn1mx7uTxTzIK7RdmHa21G6vqsUlekuTJSZ6a5HNJLk1ySWvtloHGCwAAXRskyLfWLk5y8ZRt9+XAqSRXe/xTSZ41xLgAAGC72kpfdgUAAKYkyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0aLAgX1UPqqo3VtVnq+rOqtpXVa+qqgccQh/7qqqt8XPzUGMFAIDeHT5EJ1V1SpJrkzwwyTuTfCTJ9yT5hSRPrqontta+OGV3tyZ51SrrbxtgqAAAsC0MEuST/GZGIf65rbXXLK+sqt9I8vwkL03y7Cn7+nJr7eKBxgUAANvSzIfWVNXJSc5Osi/J61Y8/OIktyd5ZlXtmnVbAADAyBB75J80Xl7ZWrt78oHW2v6qel9GQf/xSd49RX/3raqfSvLgjD4EfCjJe1prdw0wVgAA2BaqtTZbB1UvT3Jhkgtba7++yuOvTXJBkue01l6/Tl/7kjxklYc+keRZrbWrpxzT3jUeOu3UU0896rLLLpumm8Fd95lbN/S8448cLf/+jgEHswGnn3jM3Lex0RpNa+Vr2L9/f5JkaWlpkP7nPf7NMNTvea3abkaN5j1XF/17HuI9ofcazWP8K+dsj69hs01bo43O2e34/87Qhv5/bDWLfs+b1UZ/B7PW9vzzz88NN9zwgdbaGRvqYApDnLVmuTpr/ZaX199/ir7elOSsJCck2ZXkkUnekOSkJH9UVY/a8CgBAGAbGerLrgdT4+W6u/5ba5esWHVdkmdX1W1JXpDk4iQ/NkU/q37yqaq9S0tLj9m9e/d6XczFeRddsaHnXfjIrydJXvG/NuPXtbZ95+6e+zY2WqNprXwNe/bsSZIMNSfmPf7NMNTvea3abkaN5j1XF/17HuI9ofcazWP8K+dsj69hs01bo43O2e34/87Qhv5/bDWLfs+b1UZ/B7PWdp5/JVk2xB755T3ua/3d4ugV7Tbit8bL75+hDwAA2DaGCPIfHS8fusbjp46XH5thG58fL535BgAAMkyQv2q8PLuq7tFfVS0leWKSO5K8f4ZtPGG8/LsZ+gAAgG1j5iDfWrsxyZUZfSH1ghUPX5LRXvS3tNZuT5KqOqKqThtfDfYbquoRVXXsyv6r6iFJXju++9ZZxwsAANvBUN+efE6Sa5NcWlVnJbk+yeOSnJnRITUvmmh74vjxmzIK/8vOSXJRVV2V0ekm9yc5JckPJblfknclecVA4wUAgK4NEuRbazdW1WOTvCTJk5M8Ncnnklya5JLW2i1TdHNVkocleXRGh9LsSvLlJNckuTzJ5W3Wk94DAMA2Mdj5DFtrn0ryrCna7cuBU1JOrr86yVQXfAIAgJ1uiC+7AgAAm0yQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOiQIA8AAB0S5AEAoEOCPAAAdEiQBwCADgnyAADQIUEeAAA6JMgDAECHBHkAAOjQYEG+qh5UVW+sqs9W1Z1Vta+qXlVVD1hEPwAAsJ0dPkQnVXVKkmuTPDDJO5N8JMn3JPmFJE+uqie21r64Wf0AAMB2N9Qe+d/MKHw/t7X29NbaRa21JyV5ZZKHJXnpJvcDAADb2sxBvqpOTnJ2kn1JXrfi4RcnuT3JM6tq12b0AwAAO8EQe+SfNF5e2Vq7e/KB1tr+JO9LclSSx29SPwAAsO1Va222DqpenuTCJBe21n59lcdfm+SCJM9prb1+3v2M2+5d46FH3fe+973Pgx/84IM9fW6+9k93beh5h48/bn397oO3m7f7HXGfuW9jozWa1srXcPfdo6IedtgwR5nNe/ybYajf81q13YwazXuuLvr3PMR7Qu81msf4V87ZHl/DZpu2Rhuds9vx/52hDf3/2GoW/Z43q43+Dmat7Sc/+cnceeedt7TWjttQB1MY4suux4yXt67x+PL6+29SPwdz15133nnrDTfcsG+GPhbhtPHyIwsdxfaktvOjtvOjtvOhrvOjtvOjtvMza21PSvKVYYayukHOWrOOGi9n2/V/CP201s6YcVtbyvJfGLbb69oK1HZ+1HZ+1HY+1HV+1HZ+1HZ+eqjtEH+HWd5Tfswajx+9ot28+wEAgG1viCD/0fHyoWs8fup4+bFN6gcAALa9IYL8VePl2VV1j/6qainJE5PckeT9m9QPAABsezMH+dbajUmuzOiA/gtWPHxJkl1J3tJauz1JquqIqjptfBXXDfcDAAA72VBfdn1OkmuTXFpVZyW5PsnjkpyZ0aEwL5poe+L48ZsyCu0b7QcAAHasmc8j/42Oqr49yUuSPDnJcUk+l+S/J7mktXbLRLuTknwiyU2ttZM22g8AAOxkgwV5AABg88zvMmAAAMDcCPIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkt4Cq+raq+pGquriq/rCqPldVbeLnpDls86QV25jm54eHHse8LaK2K7Z/36r6uaq6sqo+XVV3jsfwnqp6XlUdPc/tb4aq+q6qenVV/W1V3VpVt1XVx6rqP1XVEwfeVrfztqpOrqp/X1V/XVW3VNVXq+rGqvqdqnrqnLZ5WFX971X1zqq6qaq+VlWfr6q/qKoXVdXx89juZtvM2m5g/v38kNvfTFX1zVX1r6rqhVX19vEcmnxtu+e47W07dxdR150wb2vk8VX1K1X1rqraN34v+Nr4/93/Ma75CXPa/mLmbGvNzwJ/krwsSVvn56Q5bPekKba78ueHF12vHmo7sf3Tk1y3zvZvSvK9i67VBl9fJfl3Sb6+zmt8Q5IjdvK8zeiq1V9dZ5zvSHL0gNt8UJJr1tnmF5P82KLr01NtNzD/fn7RNdrg63z2FK9t95y2vW3n7qLqut3nbZKnJfnMlK/tjiQXbpc5e3hYtPutsu6WJMdu4hj+Lsnrp2j34XkPZGALq22N9vT/aZLlT/5fTvL2JPuSfGuSHx8/9uAk76qqH2itfXDe4xrYryX5pYn71ya5OqNg/7gkP5hR2D8/yVFV9dNt/I43kC7mbVU9O8nrJlZ9KMkfZxQ+H5XkR5IcnuTpSd5RVU9prf3jjNt8QJIrkzx8vOqOJP8tyUczmv9PS3Ly+PbvVtXTWmt/NMs2F2ERtZ3wpST/fop21wy0vc222vvn/vH6I+a10R0wdxdS1wnbdd5+Z5Jvm7h/Y5L3JflkRnPoOzJ6Pzg+o1q/vKq+tbX2glk3vPA5u+hPUTv9J8mLxxPg15L8RJLvGK+f/BR30hy2e9JE/3sWXYftVNvxNt49sY1rkhy34vEjM9pLuNzmb5PcZ9E1O4TX930TY/96knNXafMvk9w20e4nB9huV/M2ySlJ7pwY80WrtHl0kpsn2vzyANv9zxP9Xb9ynie5T0YBeLnN5zPgXwO2eW2X+9q36BrMub4/m2RPkl9P8n8keVhGH8z3TdRg9xy2u63n7gLruq3nbZKLknxlXNfT12hzvyS/tSIDnDnAthc6ZxdefD9rTgxBvt/a/uBE/19Mcvwa7Y5KcsNE23+z6Nocwmt838S4X3qQdudPtPtEksNm3G5X8zbJ2ybG+7aDtDt7ot2tSe4/wzZPS3LXuK9/TPKINdodluS9E9t9yaLrtdVrO+5vWweiKV7/3ALnTpm7m13Xcf/bet4m+a4kx07Z9l0T9Xj7jNtd+Jz1ZVcY3k9P3H5Da+3vV2vUWvtqkles8bwtq6pOSfIvxndXvoaV3pjk0+PbJyX5/vmNbGupqm/K6BCqZPzGvVbb1tqVSd4/vnt0RoeCbNS5OXAig99rrf3tGtu8O8n/M7HqmTNsc1MtsLbM17afu8xHa+1DrbVbpmx+6cTtJ8y46YXPWUEeBlRVh2V0HN6y313nKb+XURBJku+rqm+ey8CG9aMTt/+0tfaltRq21r6e0SFEy35sbqPaes7OgeNhP9Ra++g67f/rxO1Z6jT5+1lv/v1pRt/fSJKTqurRM2x3My2qtszXTpi7LN4nJm4fN2NfC5+zgjxJcmpVvXt8mqR/rKovVNXfVNXrauDTB+4AJyc5Znx7f0ZfvltTa+2LGR1Tl4z+PX7X/IY2mMdM3J7mC1Hvnbg95H+2W33ebnqdquq+GX3pa6rtjvcSXTvrdhdgK8zBY6vqivFp7f6xqr5UVddX1X8e+pSXO8EOmruLZt4mJ07c/oeNdrJV5qwgTzL6pveTknxLRt+aPy6jQPmcJNdU1Z/0fM7eTfaIids3jv/hrufjazx/q5oc4w1TtJ/X69vq83aWOn17bewaA6dl9MWqJPlya+0Lh7jdHuZfspjarrSU5KkZnX3qiCT3z6j+/ybJFVX1V1V16gDb2Sl2ytxdNPM2+dcTt9+7Zqv1bYk56/ST/FOSP0/yN0n+PqNJ+eCMzjjykHGbs5P8VVU9vrX22YWMsh+TF5r49Jqt7ulTazx/qzrU1zj5+o6tqiNaa/804xh6mLeHVKfW2peq6vYku8arjs/oLAxz2+ZYb/MvWUxtJ92d5K+SfDDJZ8f3T0yyOwdOQffYjObf97bWrpthWzvFTpm7i7Tj521VPSLJsyZWXTZDd1tizgryO9dtGe25fFtr7V7/oY2P9T4voy+F7Ery7Rkdz/0vVrblHpYmbt8+5XO+usbzt6pDfY1fXXF/KaPz+W9ET/N2o3NhOWxuZC7shPmXLKa2yy5K8qbW2udXe7CqnpbktzP6S9ExSX6/qr6ztfa1Gba5E+yUubsoO37eVtWuJG9N8s/Gq97RWrtqhi63xJwV5NdQVb+WYb8U9eLW2n8ZsL+ZjP8EtObFdMaHhLyxqj6V0QVWDkvyhKr60dbaO2fZ9jav7ZETt6e98MzkG+VRs2x8k2p7qK9x5X8ER2WDQX6R83YDFjEXFjr/NtHCXmdr7T+s8/jvV9W/zOhY2F0ZXYjm/0zy6o1uc4fYKXN3IXb6vB3v5Lk8yXePV306yc/N2O2WmLOC/Nq+NaMLNQzlAQP2tWlaa39aVf9fRheuSEbHls0aiLZzbe+YuP3P1mx1T5NX+lu59/pQbUZt78iBPZvTvMaVVzKc9TWua07z9lAtYi4sev5tli39OltrH6qqVyd54XjVv842CURztKV/pzvBdp23VVVJ3pADO7m+kuTp45NNzGJLzFlfdmUak6dUmvWcq9vd/onbu9ZsdU+Tn8r3r9lq6zjU17hyr8NmvcZFz9tFzIWdMP+SPl7n5Pz7nvEeQdbWw+90J9iO8/Y1GV1RNxnNkye31vYO0O+WmLP2yK+htXZeRsfacs9vWc98FpBtXtubJ26fuGare3rQxO1VLx41rU2q7c058CWdE5Os94Y4+fq+NMAXXac16LzdgEOaC1V1TO75n8Gqx7IOuc2xwebfJlpEbQ/V5Pw7PKOzKm34VHc7wE6Zu1vdtpq3VfWaJBeM796W5CmttT8fqPstMWe3wyct5q+tcZt7+/DE7VOm3Jvxzydur3pVuC1m8jU+dIr2i3p9i563h1qnyVO+fbq1dusGtvnRjC4XniQPmPICY73Nv2QxtT1UK+ec986D2ylzd6vbNvN2HOJ/fnz39iRPba29b8BNbIk5K8gzjcmJtxl7snp2Y5LlkHB0kkcerHFVHZsDF5S4O6PTKW51H5i4Pc2Fl75v4vYHBx7LwSx63m56ncZnmLh+YtVBtzv+oDl5Rp/N/P3Mooc5ODn/7srGz9S0I+ygubvVbYt5W1WvzYEQ/9UkP9Ram+Wc8feyVeasIM80zpm4/RcLG0UHxmdN+cOJVees1XbsGUlqfPuaKS8osWiTXxr9waq6/1oNq+o+SZ4+seodcxrTahY9b/8kB85Q8KiqWm/P8U9M3J6lTpO/n/Xm31k58IXmm1prHzhY4y1kUbU9FJO1/6spLw630+2EubvVdT9vxyF++XCa5RB/9Zw2t/A5K8hzUFX1pBw480eS/NdFjaUjl0/cfnZVPXC1RlV1ZJJfnFj1lrmOaiCttY8nef/47q4kLzhI82dldKGmJLkpyXvmOLRv2ArztrV2W5L/vjykJL+6Vtvxad+W99Tsn3jeRrwto7/uJMk5VfXw1RqN9w79ysSqy1drtxUtsLZTGV905nkTq7xvTmfbz92tbDvM26p6Xe4Z4n+4tbZnjptc/JxtrfnZgj8ZHZe2/HPSITzvvInn7VujzUOSvDvJU5Icvkabw5L8TEb/8S339xdJatG12cq1nWh71UTb9yY5dsXjRyZ5+0Sb69f6XWzFnyQ/MDH2f0ryk6u0OWvF/Dl3ltr2OG8z+jP1P06M5RdXafOojK6yuNzmRWv0tXty7q6z3TdNtP1wkoesePw+GZ3JYbnNPyQ5ZtHzaqvXNsn/zGiv2/0O0uaHMvoS23J/NyU5ctH1GrDu+yZe2+4pn2PuLrCuO2XeJnndxPhvT3LmDH11M2drvBEWpKq+Jatf4OYZE7f/KPc+3+hr2yqfMqvqvIwmVTL6081Jq7Q5KcknxndvyWjv6vVJvpRREPr2JD+YZPK5n07yhNbatJchXrhF1Hai7ckZXVhj+WwpX8roCqM3ZXTGl2dkdM73ZPRN+t1tmNNhbZqqenmSCydWXZPk6oyOq3xckrNz4LCh/zejIL/qG852nrdVdUGS106s+puMLlb11YyC5o8kOWL82NVJ/lVr7c5V+tmd0QfEJElrrVa2mWh7XJL35cA1Bb6a5L9l9OWsByT50SSnjB+7K8mPttauOLRXtnibXduqWp6/t2U0/65L8sWM9sh9W5Izc+A7L8no+zI/0Frr4bsv91JVv7fK6qfkwCn03pN7n9Hkd1trk6cwNHdX2Oy67oR5W1UvTPLSiVV/kOn/AvxfWmufWtHf7vQyZxf9CWqn/2QUOtoGfs5bo7/zJtrsG2ibf5Dk+EXXqofarmj/XRl9Oj/Ytj6V5PsXXasN1reS/FqSr6/zGn87yT9bp69tPW+T/N8ZXTzkYOP9/RxkL00OYQ/RuP2DM/owebBt3pLkGYuuTy+1PcT5974k/3zR9Zmxtht5/7zY3N1add0J8zbJmzdY15ZV/gLS05x1Hvmd6ZNJHj/x89Ak35LkmzPas/nljM6+cm2St7aOPpVvJW10lbxHZ3SoxzkZ7fH45oz2IH88o0/sv91a+8riRrlxbfTu9cvjK6j+bEaH0jwooz8jfjajvSFvaq1dM9Amu523rbXXVNW7Mrok+FMyetO/X0bnIf6LJJe3gfcqttY+WVXfm+R/S/KTGV2a/PiMDjv6REbh9j+11m5es5MObHJtH50D8+87c2D+3Tej+XdTRns8f7cNfIaMnWSnzN1NZN7O2SLnrENrAACgQ85aAwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAOCfIAANAhQR4AADokyAMAQIcEeQAA6JAgDwAAHRLkAQCgQ4I8AAB0SJAHAIAO/f/AeSVlGir0fwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 269,
       "width": 377
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "counts_scaled = pd.DataFrame(preprocessing.scale(counts))\n",
    "counts_scaled.hist(bins=25,xlabelsize=16);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.77635684e-16])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counts_scaled.mean().values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Notice that values that used to be zero have now become negative.\n",
    "\n",
    "In some situations it may not be sensible to change zeros into something else.  It may make more sense to map all values into a fixed range, for example $[0, 1]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAIbCAYAAAC5XGsnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAABYlAAAWJQFJUiTwAAAtQklEQVR4nO3dfZRlZ10n+u8vJCYhVAIJkmgQYjJJOpIYSRhehNEmGTMBRg0od8YJaFgqw4AX4Zq7jOAsEu7lyh3QQHgRuQ5GQF06OgyjAYxiOrxkUGlkAhJCjHRA3oUhVEISSfq5f5xT9Omiqut01Xl5qurzWWuvvWvv5zzPU/vps/t7du29T7XWAgAA9OGQeXcAAADYR0AHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADsCKquqEqnp1Vd1aVXdX1Req6o+r6vx59w1gK6vW2rz7AEBnqup7k/xFkuOGq76W5AEZnNhpSV7UWnv5nLoHsKU5gw7AfqrqyCT/PYNw/jdJzmytHZPkQUl+NUkl+ZWqumB+vQTYupxBB2A/VfWCJFcmuSPJjtbaZ5Ztf1uSi5J8qLV27sw7CLDFOYMOwHIXD+e/uzycD71iOD+nqnbMqE8A24aADsA3VdVCkqWz4n+6SrEPJLl9uHze1DsFsM0I6ACMOiODa8yT5G9XKtBa25vk5uGP3zOLTgFsJwI6AKO+Y2T5swcot7TtOw5QBoB1ENABGHXUyPJdByj39eH8AVPsC8C2JKADMKrWLgLANAnoAIy6Y2T5yAOUu/8K5QGYAAEdgFGj151/5wHKLW373BT7ArAtCegAjPp4kqVvsHvESgWq6pAkpw9//NgsOgWwnQjoAHxTa20xyQeHP/7QKsUek+SY4fK7p94pgG1GQAdgud8dzi+uqpUeo3jpcL67tXbzCtsB2AABHYDlfiPJbUkWkvxJVX1PMviW0ar6T0meNiz3ojn1D2BLq9ba2qUA2Faq6uwMLl85brjqaxk88/yQDK5Rf1Fr7eVz6h7AliagA7CiqjohyS8l+ddJTswgpP9Vkitba649B5gSAR0AADriGnQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB05dN4dmKWq+mSSo5PsmXNXAADY2k5K8rXW2ncf7Au3VUBPcvSRRx557BlnnHHsrBteXFxMkiwsLMy6aWbEGG8Pxnl7MM7bg3He+uY5xjfddFPuuuuudb12uwX0PWecccaxu3fvnnnDu3btSpLs3Llz5m0zG8Z4ezDO24Nx3h6M89Y3zzE+99xz86EPfWjPel7rGnQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjkwkoFfV/1tV766qT1fVXVX1lar6m6p6SVUdd5B1PbSq3lRVn62qe6pqT1W9qqoeNIm+AgBAzyZ1Bv2FSY5K8mdJXp3kd5Lcm+TyJDdW1XeNU0lVnZJkd5JnJfmrJFcm+fskP5/kfxxs2AcAgM3m0AnVc3Rr7e7lK6vqZUlelOSXkjx3jHpen+QhSZ7fWnvNSD2/lsGHgJclec5EegwAAB2ayBn0lcL50B8M56euVUdVnZzkgiR7krxu2eaXJLkzyTOr6qh1dhMAALo3qTPoq/nh4fzGMcqeN5xf21rbO7qhtbZYVe/PIMA/Nsm7D1RRVe1eZdOOxcXF7Nq1a4zuTNbi4mKSzKVtZsMYbw/GeXswztuDcd765jnGS22vx0QDelVdmuQBSY5J8qgkT8ggnL98jJefPpx/YpXtt2QQ0E/LGgEdgPn66Gdun2r9Z554zFTrB5inSZ9BvzTJ8SM/vyvJJa21L43x2qWj7WpH9aX1D1yrotbauSutr6rdCwsL5+zcuXOM7kzW0ie3ebTNbBjj7cE4j+eSy66Zav17Lt451fqN8/ZgnLe+eY7xwsLCul870eegt9ZOaK1VkhOSPC3JyUn+pqrOmUD1tdTMBOoCAIAuTeWLilprX2itvS2DS1KOS/LmMV62dIZ8tb9bHr2sHAAAbDlT/SbR1tptST6W5BFV9eA1it88nJ+2yvalJ8Gsdo06AABselMN6EPfOZzft0a564bzC6pqv35V1UKSxye5K8kHJts9AADox4YDelXtqKoTVlh/yPCLih6S5IbW2v8arj9s+JpTRsu31m5Ncm2Sk5I8b1l1V2TwTaVvbq3dudE+AwBArybxFJcLk7yiqt6T5NYkX87gSS4/mMFNop9P8rMj5U9MclOS2zII46Oem+SGJFdV1fnDco9J8sQMLm158QT6CwAA3ZpEQP/zJG/M4BKUszN4DOKdGQTqtyS5qrX2lXEqaq3dWlWPSvLSDIL/k5N8LslVSa4Ytx4AANisNhzQW2sfzbdeknKg8nuy75GJK23/dJJnbbRfAACwGc3iJlEAAGBMAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARwR0AADoiIAOAAAdEdABAKAjAjoAAHREQAcAgI4I6AAA0BEBHQAAOiKgAwBARzYc0KvquKr6map6W1X9XVXdVVW3V9X7quqnq2rsNqpqT1W1VabPb7SvAADQu0MnUMfTk/x6ks8luS7Jp5Icn+RpSX4zyZOq6umttTZmfbcnedUK6+/YeFcBAKBvkwjon0jyI0muaa3tXVpZVS9K8ldJfiyDsP5HY9b31dba5RPoFwAAbDobvsSltfYXrbU/Hg3nw/WfT/KG4Y87N9oOAABsB5M4g34g3xjO7z2I1xxeVc9I8rAkdya5Mcl7Wmv3jVtBVe1eZdOOxcXF7Nq16yC6MxmLi4tJMpe2mQ1jvD0Y5/FcetbBHPYP3rT3v3HeHozz1jfPMV5qez2mFtCr6tAkPzn88V0H8dITkrxl2bpPVtWzWmvXT6RzAADQqWmeQX95kjOTvKO19qdjvua3krw3yd8mWUxycpKfS/LsJO+sqse11v7nWpW01s5daX1V7V5YWDhn586dY3ZncpY+uc2jbWbDGG8Pxnk8l1x2zVTr33PxzqnWb5y3B+O89c1zjBcWFtb92qkE9Kp6fpJfSPLxJM8c93WttSuWrfpokudU1R3D+i5P8tQJdRMAALoz8S8qqqrnJXl1ko8leWJr7SsTqHbpZtMfmEBdAADQrYkG9Kp6QZLXZnDm+4nDJ7lMwheH86MmVB8AAHRpYgG9qn4xyZVJPpxBOP/igV9xUB43nP/9BOsEAIDuTCSgV9V/zOCm0N1Jzm+t/eMByh5WVTuq6pRl6x9RVceuUP7hGZyVT5K3TqK/AADQqw3fJFpVP5XkpUnuy+AJLM+vquXF9rTWrh4un5jkpiS3JTlppMzTk1xWVdcl+WQGT3E5JclTkhyR5B1JXrnR/gIAQM8m8RSX7x7O75fkBauUuT7J1WvUc12S05M8MoNLWo5K8tUk78vguehvaa21jXUVAAD6tuGA3lq7PIPHH45bfk+SbznFPvwSIl9EBADAtjbxxywCAADrJ6ADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdGTDAb2qjquqn6mqt1XV31XVXVV1e1W9r6p+uqoOqo2qemhVvamqPltV91TVnqp6VVU9aKN9BQCA3h06gTqenuTXk3wuyXVJPpXk+CRPS/KbSZ5UVU9vrbW1KqqqU5LckOQhSd6e5ONJHp3k55NcWFWPb619eQJ9BgCALk0ioH8iyY8kuaa1tndpZVW9KMlfJfmxDML6H41R1+szCOfPb629ZqSuX0vywiQvS/KcCfQZAAC6tOFLXFprf9Fa++PRcD5c//kkbxj+uHOteqrq5CQXJNmT5HXLNr8kyZ1JnllVR220zwAA0KtJnEE/kG8M5/eOUfa84fzaFcL+YlW9P4MA/9gk7z5QRVW1e5VNOxYXF7Nr164xujNZi4uLSTKXtpkNY7w9GOfxXHrWOIf99Zv2/jfO24Nx3vrmOcZLba/H1AJ6VR2a5CeHP75rjJecPpx/YpXtt2QQ0E/LGgGdzeujn7l9qvWfeeIxU63/7m/szW2b/HfYCqb57+j4I5MjDvMALNgqpn3cdsxmPaZ5Bv3lSc5M8o7W2p+OUX7pX/Bq75Kl9Q9cq6LW2rkrra+q3QsLC+fs3LlzjO5M1tInt3m0vZlcctk1U61/z8U7p1b3rl27cttnbs8rPzLdP0xN83fYKqb57+jSs+7NmccueC+vYTO/lxPH7O1iFsdtx+z5mud7eWFhYd2vncppoKp6fpJfyOApLM+cVLXD+ZpPgwEAgM1q4gG9qp6X5NVJPpbkia21r4z50qUz5Kv9LejoZeUAAGDLmWhAr6oXJHltko9mEM4/fxAvv3k4P22V7acO56tdow4AAJvexAJ6Vf1ikiuTfDiDcP7Fg6ziuuH8guXfPlpVC0ken+SuJB/YYFcBAKBbEwnoVfUfM7gpdHeS81tr/3iAsodV1Y7ht4Z+U2vt1iTXJjkpyfOWveyKJEcleXNr7c5J9BkAAHq04duWq+qnkrw0yX1J3pvk+VW1vNie1trVw+UTk9yU5LYMwvio5ya5IclVVXX+sNxjkjwxg0tbXrzR/gIAQM8m8Vyh7x7O75fkBauUuT7J1WtV1Fq7taoelUHgvzDJk5N8LslVSa44iBtOAQBgU9pwQG+tXZ7k8oMovyf7Hpm40vZPJ3nWRvsFAACbka/DAwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB2ZSECvqh+vqtdU1Xur6mtV1arqreuoZ8/wtStNn59EXwEAoGeHTqieX05ydpI7kvxDkh0bqOv2JK9aYf0dG6gTAAA2hUkF9BdmEMz/LskPJrluA3V9tbV2+SQ6BQAAm81EAnpr7ZuBvKomUSUAAGxLkzqDPkmHV9UzkjwsyZ1JbkzyntbaffPtFgAATF+11iZbYdXODC5x+Z3W2jMO8rV7kjx8hU2fTPKs1tr1Y9aze5VNO0499dT7v/GNbzyYbk3E4uJikmRhYWHmbW8mH/3M7VOt/8wTj5la3YuLi7n7G3vzhbum1kSS6f4OW8U0/x0df2RyxGGHeC+vYTO/lxPH7O1iFsdtx+z5mud7+dnPfnZuueWWD7XWzj3Y1/b2mMXfSnJ+khOSHJXkrCS/keSkJO+sqrPn1zUAAJi+ri5xaa1dsWzVR5M8p6ruSPILSS5P8tQx6lnxk0pV7V5YWDhn586dG+zpwdu1a1eSZB5tbyaXXHbNVOvfc/HOqdW9a9eu3PaZ2/PKj0z3bTXN32GrmOa/o0vPujdnHrvgvbyGzfxeThyzt4tZHLcds+drnu/ljZy17+0M+mreMJz/wFx7AQAAU7ZZAvoXh/Oj5toLAACYss0S0B83nP/9XHsBAABTNvOAXlWHVdWOqjpl2fpHVNWxK5R/eJLXDn986yz6CAAA8zKRuyKq6qIkFw1/PGE4f1xVXT1c/sfW2qXD5ROT3JTktgyezrLk6Ukuq6rrMnis4mKSU5I8JckRSd6R5JWT6C8AAPRqUrctf1+Sn1q27uThlAzC+KU5sOuSnJ7kkRlc0nJUkq8meV+StyR5S5v0Q9sBAKAzEwnorbXLM3gE4jhl9ySpFdZfn2SsLyICAICtarPcJAoAANuCgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQEQEdAAA6IqADAEBHBHQAAOiIgA4AAB0R0AEAoCMCOgAAdERABwCAjgjoAADQkYkE9Kr68ap6TVW9t6q+VlWtqt66zroeWlVvqqrPVtU9VbWnql5VVQ+aRF8BAKBnh06onl9OcnaSO5L8Q5Id66mkqk5JckOShyR5e5KPJ3l0kp9PcmFVPb619uWJ9BgAADo0qUtcXpjktCRHJ/kPG6jn9RmE8+e31i5qrV3WWjsvyZVJTk/ysg33FAAAOjaRgN5au661dktrra23jqo6OckFSfYked2yzS9JcmeSZ1bVUevuKAAAdK6nm0TPG86vba3tHd3QWltM8v4k90/y2Fl3DAAAZmVS16BPwunD+SdW2X5LBmfYT0vy7gNVVFW7V9m0Y3FxMbt27VpXBzdicXExSebS9mZy6Vn3TrX+ae7/xcXFHH/k5v4dtoppjsHxRw7G2jgc2GZ/Hzhmbw+zOG77NzRf83wvL7W9Hj0F9GOG89tX2b60/oHT78p03P2NvbntM6v9ev0788Rj1i7E1H10yv+Gpj3O0+7/VjCLfeT9vLZpH7O3whhs9uMR49nMx+3jj0yOOKynC0bG01NAX0sN52te595aO3fFCqp2LywsnLNz585J9mssu3btym2fuT2v/Mhm2uX723Pxzqm3ccll10y1/mn+DlthjJPpj/O0x3jaLj3r3px57EKmeRyZxT7a7OM87f7P4v08i2PqtBnntRnn+ZrFMXs1CwsL635tTx8plj6erfZx+ehl5QAAYMvpKaDfPJyftsr2U4fz1a5RBwCATa+ngH7dcH5BVe3Xr6paSPL4JHcl+cCsOwYAALMy84BeVYdV1Y7ht4Z+U2vt1iTXJjkpyfOWveyKJEcleXNr7c6ZdBQAAOZgIndFVNVFSS4a/njCcP64qrp6uPyPrbVLh8snJrkpyW0ZhPFRz01yQ5Krqur8YbnHJHliBpe2vHgS/QUAgF5N6rbl70vyU8vWnTyckkEYvzRraK3dWlWPSvLSJBcmeXKSzyW5KskVrbWvTKi/AADQpYkE9Nba5UkuH7Psnux7ZOJK2z+d5FmT6BcAAGw2Pd0kCgAA256ADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANCRiQX0qnpoVb2pqj5bVfdU1Z6qelVVPegg6thTVW2V6fOT6isAAPTq0ElUUlWnJLkhyUOSvD3Jx5M8OsnPJ7mwqh7fWvvymNXdnuRVK6y/YwJdBQCArk0koCd5fQbh/PmttdcsrayqX0vywiQvS/KcMev6amvt8gn1CwAANpUNX+JSVScnuSDJniSvW7b5JUnuTPLMqjpqo20BAMBWN4kz6OcN59e21vaObmitLVbV+zMI8I9N8u4x6ju8qp6R5GEZhPsbk7yntXbfBPoKAABdq9baxiqoekWSS5Nc2lr71RW2vzbJ85I8t7X262vUtSfJw1fY9Mkkz2qtXT9mn3avsmnHqaeeev83vvGN41QzUYuLi7n7G3vzhbtm3vTEnHniMVNv46OfuX2q9U/zd9gKY5xMf5ynPcbTdvyRyRGHHZKFhYWptTGLfbTZx3na/Z/F+3kWx9RpM85rM87zNYtj9mqe/exn55ZbbvlQa+3cg33tJJ7isvQvb7XRW1r/wDHq+q0k5yc5IclRSc5K8htJTkryzqo6e929BACATWBSN4keSA3na56qb61dsWzVR5M8p6ruSPILSS5P8tQx6lnxk0pV7V5YWDhn586da1Uxcbt27cptn7k9r/zILHb5dOy5eOfU27jksmumWv80f4etMMbJ9Md52mM8bZeedW/OPHYh0zyOzGIfbfZxnnb/Z/F+nsUxddqM89qM83zN4pi9mo2ctZ/EGfSlM+Sr/Q3n6GXl1uMNw/kPbKAOAADo3iQC+s3D+WmrbD91OP/EBtr44nDuSTAAAGxpkwjo1w3nF1TVfvVV1UKSxye5K8kHNtDG44bzv99AHQAA0L0NB/TW2q1Jrs3gRs7nLdt8RQZnvd/cWrszSarqsKraMfz20W+qqkdU1bHL66+qhyd57fDHt260vwAA0LNJ3RXx3CQ3JLmqqs5PclOSxyR5YgaXtrx4pOyJw+23ZRDqlzw9yWVVdV0Gj1VcTHJKkqckOSLJO5K8ckL9BQCALk0koLfWbq2qRyV5aZILkzw5yeeSXJXkitbaV8ao5rokpyd5ZAaXtByV5KtJ3pfkLUne0jb60HYAAOjcxJ4r1Fr7dJJnjVFuT/Y9enF0/fVJxvoiIgAA2KomcZMoAAAwIQI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcmFtCr6qFV9aaq+mxV3VNVe6rqVVX1oHnUAwAAm9Ghk6ikqk5JckOShyR5e5KPJ3l0kp9PcmFVPb619uVZ1QMAAJvVpM6gvz6DUP381tpFrbXLWmvnJbkyyelJXjbjegAAYFPacECvqpOTXJBkT5LXLdv8kiR3JnlmVR01i3oAAGAzm8QZ9POG82tba3tHN7TWFpO8P8n9kzx2RvUAAMCmVa21jVVQ9Yoklya5tLX2qytsf22S5yV5bmvt16ddz7Ds7lU2nX344Yff72EPe9iBXj4Ve/fuzd7Wcu/etcv26ojD7jf1Nu7+xn1TrX+av8NWGONk+uM87TGetkMPSQ6pyiGHTO8hWLPYR5t9nKfd/1m8n2dxTJ0247w24zxfszhmr+ZTn/pU7rnnnq+01o472NdO4ibRY4bz21fZvrT+gTOq50Duu+eee26/5ZZb9mygjvXaMZx/fA5tMxvGeHswztuDcd4ejPPWN88xPinJ19bzwok8xWUNNZxv7FT9QdTTWjt3g21N3NJZ/R77xmQY4+3BOG8Pxnl7MM5b32Yd40mc7186s33MKtuPXlZu2vUAAMCmNYmAfvNwftoq208dzj8xo3oAAGDTmkRAv244v6Cq9quvqhaSPD7JXUk+MKN6AABg09pwQG+t3Zrk2gwuhH/ess1XJDkqyZtba3cmSVUdVlU7ht8auu56AABgK5rUTaLPTXJDkquq6vwkNyV5TJInZnBJyotHyp443H5bBmF8vfUAAMCWs+HnoH+zoqrvSvLSJBcmOS7J55L8tyRXtNa+MlLupCSfTHJba+2k9dYDAABb0cQCOgAAsHGz/1olAABgVQI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcE9AOoqpOr6v+pqg9X1Veq6utVdWtV/W5VPXlKbR5SVf+2qt5eVbdV1d1V9cWq+suqenFVHT+NdrezWY5zVX17VV1cVb9ZVR+sqi9X1Teq6qtV9bGqurqqLqyqmmS7zOf9vEo/fr+q2sh09aza3g7mOc5V9S+q6qqqurGqvlRV91TVP1TVX1fV66rqx6vqqGn2YTuY0//N319Vr6+qDw3b/EZVLVbV31XVH1XVT1bVEdNoe7upqgdX1b+qqhcN9+1ty46ZO6fYdj8ZrLVmWmHK4FtNv56kHWB6W5KjJ9jmQ5O8b402v5zkqfPeP1tlmuU4J3lnknvXaGtp+h9JTp33/tkq0zzez6v042krtHv1vPfPVpnmNc5JHp7kj8d8b++c937azNOsxzjJ0Un+65hj+8kkT5j3PtrMU5LnzOs91FsG80VFK6iq5yT59ZFVNyZ5VwYHhbOT/HCSQ4fb/iLJk1pr/7TBNh+U5P1JzhiuuiuDg8LNSY5N8iNJTh5uuzfJj7TW3rmRNre7WY9zVd2d5PDhj19P8pdJPpzkH5M8IMnjkuwceckXMjjY/91622Q+7+dV+nFckr9NcnySf0rybcNNv91au2TS7W038xrnqtqR5N1JvnO46utJdiX5SJKvJXlwku9I8tgkJyV5Ymtt10bb3Y7mcMy+X5L3ZnBsXvLhDE6gfDaDbzs/M8n5SZb+6nl3ku9vrf3NetvdzqrqBUmuXLZ6MckRSQ4b/jzx91CXGWzen5Z6m5KckuSe7Pu0dNkKZR6Z5PMjZX5pAu3+55H6bkpy0rLt90vyupEyX8yUz/Zt5Wke45zBgfuGJM9Icv9VyjwuyedG2rx+3vtqM0/zej+v0pffG3nvvnakvavnvZ82+zTH4/YDk3x6pM7/nOTBByh/RpJvn/f+2ozTnI7ZPztS151JLlql3NlJbh0p++fz3l+bdUryMxl8wP3VJP8uyekZfPjZM7J/d06h3e4y2NwHo7cpye+MDMDvHKDcBSPlbk/ywA20uSPJfcO6/inJI1Ypd0gGn+aX2n3pvPfXZp3mNM47xyz32CR7R9o9a977a7NO8xjnVeq/aKT+n0hy+cjPV897P232aV7jnOTqkfqunPd+2MrTnI7Z7xqp64o1yv7LkbL3JTli3vtsK03TDOi9ZjA3iY6oqgdkcI1oMtz5q5VtrV2b5APDH4/O4D/g9bo4+27Y/cPW2t+u0ubeJP/3yKpnbqDNbWte49zG/JNca+0DSf5qZNXjVivL6ub4fl7ej2Oz78/yf9Ja+71J1c38xrmqTk7yk8MfP5nkF9dbFwc2x/fySSPLN6xR9r0jy4ckedAG2mW2usxgAvr+LsjgOqckubG1dvMa5f/LyPJTN9Duj44s/8EaZf8syVeHyydV1SM30O52Na9xPhifHFk+bkZtbjW9jPNVSU7I4Hrk/zDBehmY1zj/++y77vj1bQr3LfBN8xrju0aWH7xG2W8fWf56BvcWsTl0mcEE9P2dM7L8vjHKj35iXtcgVdXhSb5n3HaHn+BGP8kL6Adv5uO8DieOLH9pRm1uNXMf56r64QzOziTJL7bW/mES9bKfeY3zk0aW31lVh1fVc6vqvcNHLN5dVZ+uqrdV1b+rKv/frt+8xni0np+rqm9bteT+f0H5vdbaNzbQLjPScwZzwNjfI0aWbxmj/OjTNb6rqo5eR5s7Mrj5IEm+2lob51P3aLuPWLUUq5nHOI+tqk5M8v0jq967WlkOaK7jPHwqwG8Mf3zPyDKTNfNxHl5ysdTuPRlct/rBDG4ie0IGZ1sPz+CxbRdlcP30B6vqYQfbFknm917+tQyeIJIM7g3666r6iar67uEHsu+sqh+qqnck+blhub9Mcuk622P2us1gh65dZFs5YWR5zTNdrbX/VVV3Jln64onjM/gz9tTaHPr0Kq9nPPMY54Pxiuw7YFw/xp9zWdm8x/nVGTxe7+4kP9uGdxkxcfMY51Oy7wTXYpJ3JPlnGdzc/c4MwnpLcm6SJ2fwfn5kkvdX1TmtNX8VOzhzeS+31vZU1fkZPFf9xCTfm+R3Vyn+qSRvSvIrLnfaVLrNYAL6/hZGlu8c8zVfz76DwMKBCk64zZVez3jmMc5jqaqLM3jKRzJ41uovTKutbWBu41xVT8m+G4iuaK19Yr11saZ5jPPoDYAPHk5fSvKvW2ujN3inqs5Nck0GIfGhSV6f5OnraHM7m9t7ubX211V1ZpJnJXlJkmNWKLY3gxv7rxfON51uM5hLXPZ35MjyuG+yu0eW779J2tzuutznVfW4JL85suqXW2u7p9HWNjGXca6qByZ54/DHv0nyyvXUw9jmMc4PWGHdxcvDeZIM38M/MbLqx6rqtHW0uZ3N7ZhdVUckeWGSX84gnN+Y5FVJXpTkV5L8eQY3C/94kuuq6o3DLzhic+gyDyTOoC83esf2gW4GGXXEyPLXVy3VV5vbXXf7vKq+N8mfjLTz1iT/adLtbDPzGucrM/hWyXuT/HRr7d511sN45n3cTpIPtdb+bLXCrbXrquoDGVzHXBlc9uKvKuOby3u5qo7J4Kkd/zyDUPZvW2u/v0K578vgMpiTMvhyo8X46+dm0V0eWOIM+v4WR5aPWrXU/kY/PS2uWqqvNre7rvZ5VT0ig7Mwxw5X/X6SS1yzvGEzH+eqelKSS4Y/vrL5uu9ZmPdxOxl8rfxaRsucs2opVjKvY/arMwjnSfJ/rBTOk6S19uEMvgr+vuGqF1TVd6+zTWarqzwwSkDf3+dHlk9ctdTQ8NP16IB+cdptDj10ZPkL62hzu5vHOK9W9/dk8B/30jN0/zDJM1pr963+KsY0j3FeurTlliRXrOP1HLx5H7eT8W4uGy2z1jO12d/Mx7iqjk/yjOGPi9n/8sNv0Vr7SAYnWpJBtrroYNtkLrrNYAL6/j42sjzONYKnjiz/Q2vt9nW0eXP2fep+UFWNc+D+ZyPLK37jFQc0j3H+FsNwfl2ShwxX/dckP+GSiImZxzgvHbhPTXJXVbWVpgxuNlvyU8u2cXBmPs6ttU/l4J8K0lZZZm3zeC8/OvuepvXxMZ9rfuMqfaBf3WYwAX1/HxpZfvwY5f/FyPK6/pTdWrs7yU3jtjv8sovRZ2T7E/rBm/k4Lze8rGU0nP+3DK5vFM4nZ+7jzEzMa5xH2x3nzNvoWTePWTw48xjj0adzjPuBqkaW966zXWao5wzmJtH9/WkGN4IckeTsqjptjcej/fjI8ts20O7bk5w5XH768OfVnJ99j/i6rbX2oQOUZWXzGuck3wznf5F94fztSf433zw3cfMY598es9z3JTl7uHxrxvt2RFY2r/fz25LsHC6fN0b50TLf8rQXDmgeYzz6hTU7quqwMY7R3zuy/Ll1tsvs9ZnBWmumkSnJ72XwabklecsByv3LkXJfS/KgDbR5RgZ/YmkZfCvdGauUOyTJ9SPt/l/z3l+bdZrHOA/rOzOD6yGX6nx7ksPmvT+26jSvcR6jX5ePtHf1vPfTZp/mdNw+PoMnOCzV90MHKPvEkXL/lOS75r3PNts06zHO4Kb9e0fqevYa5b93WfknzHufbaUpyZ6RfbtzwnV3mcHmvtN7mzK4tuifRgbg/1yhzNlJPjtS5sWr1LVzpExbo93fGin7sSQPX7b9fkleM1LmS0mOmff+2qzTPMY5g68EXh7Ov23e+2IrT/N6P4/Rr8tH6rp63vtps09zPG6/dKTsF5L88xXKnJPBjWhL5d4w7/21Gac5HbP/aKTc15P82CrlHpn9A+SHkxwy7322laasI6Af5Hu5uwxWw4YZUVXPS/LakVX/M8m7MniDnp3kh5McNtx2fZJ/1Vq7Z4V6dmZwnXGSpLVWy8uMlD0uyfuTnD5c9fUMbhq8OYM/p/xoBl8vnQw+6f1oa+2ag/vNGDXLcR5+ec3N2XdZy2KSl2e8L0b4dFvl8V6sbR7v5zH6dHn23Sj62621S9ZbFwNzOm4fMWzjB4er9iZ5R5IPZvCf+LlJnpJ9NxvemOT7W2vjfmMhI2Y9xlV1UpIPZPDXkiUfzuASxS9lcJ36ozO47GGpjjszCJAfPIhfjRFV9YcrrH5S9j3e8D351vs4/qC19gfL6tmZzZzB5v2pqNcpyf+ewQPs2wGm/54DfILKQZ5xS/KwJDes0eZXssqneFO/45zBF1gcqI0DTbvmvZ82+zSP9/Ma/bl8pK6r571/tso0p+P20Un+yxjv43dmypdObYdp1mOcZEf2feBaa7o1yWPnvY82+7TO/ycv38g4D8t3lcHcJLqK1tprquodGXwr2JMyGLgjMvhT5V9mcA3cRD89tdY+VVVPSPJvMvhq6O/L4JP7YpJPZnDQ+f9aa8ufwcs6zWOcmT3jvD3M6bj9tSRPr6rzkjwzyROSfEcG16t+PoOzcm9prV07yXa3q1mPcWvt41X1mCQXZnAD4aMyeGrPQgYfFL6QZHcGlyz+YWtt3K+LpzO9ZTCXuAAAQEc8Bx0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHRHQAQCgIwI6AAB0REAHAICOCOgAANARAR0AADoioAMAQEcEdAAA6IiADgAAHfn/AXxaEtovmKy8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 269,
       "width": 372
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "min_max_scaler = preprocessing.MinMaxScaler()\n",
    "counts_minmax = min_max_scaler.fit_transform(counts.values.reshape(-1,1))\n",
    "counts_minmax = pd.DataFrame(counts_minmax)\n",
    "counts_minmax.hist(bins=25,xlabelsize=16);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__Example Application of k-means.__\n",
    "\n",
    "Here is a simple example of how $k$-means can be used for data compression.\n",
    "\n",
    "Consider the following image.  Each color in the image is represented by an integer.  Typically we might use 24 bits for each integer (8 bits for R, G, and B).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-annie19980405.jpg\" alt=\"Figure\" width=\"350px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Now cluster the pixels by their color value, and replace each pixel by its cluster center.\n",
    "\n",
    "Because there are a smaller number of colors used, we can use fewer bits for each pixel."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here, we use 4 bits (16 colors) for a compression ratio around 6$\\times$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-annie_016.png\" alt=\"Figure\" width=\"500px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here, we use 3 bits (8 colors) for a compression ratio around 8$\\times$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-annie_008.png\" alt=\"Figure\" width=\"500px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here, we use 2 bits (4 colors) for a compression ratio around 12$\\times$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-annie_004.png\" alt=\"Figure\" width=\"500px\">\n",
    "    \n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Finally, we use 1 bit (2 colors) for a compression ratio around 24$\\times$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center>\n",
    "    \n",
    "<img src=\"figs/L6-annie_002.png\" alt=\"Figure\" width=\"500px\">\n",
    "    \n",
    "</center>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "rise": {
   "scroll": true,
   "theme": "beige",
   "transition": "fade"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}