Experiments.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Experiments\n",
    "\n",
    "This notebook documents the use of the Experiment class for running many experiments, and comparing the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "ConX, version 3.7.5\n"
     ]
    }
   ],
   "source": [
    "import conx as cx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we create an experiment object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = cx.Experiment(\"XOR-Test1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Experiment takes a function, and any number of variations of values.\n",
    "\n",
    "The function takes whatever argumnets you wish, creates a network, trains it, and returns the network and a category describing the run:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def function(optimizer, activation, **options):\n",
    "    category = \"%s-%s\" % (optimizer, activation)\n",
    "    print(\"category %s running...\" % category)\n",
    "    net = cx.Network(\"XOR\", 2, 2, 1, activation=activation)\n",
    "    net.compile(error=\"mse\", optimizer=optimizer)\n",
    "    net.dataset.append_by_function(2, (0, 4), \"binary\", lambda i,v: [int(sum(v) == len(v))])\n",
    "    net.train(report_rate=10000, verbose=0, plot=False, **options)\n",
    "    return category, net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we run a number of trials (perhaps just 1). Notice that each argument should be given as a list. The total number of runs per trial is the product of the lengths of the arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "category adam-sigmoid running...\n",
      "category sgd-sigmoid running...\n",
      "category adam-relu running...\n",
      "category sgd-relu running...\n",
      "category adam-sigmoid running...\n",
      "category sgd-sigmoid running...\n",
      "category adam-relu running...\n",
      "category sgd-relu running...\n"
     ]
    }
   ],
   "source": [
    "exp.run(function,\n",
    "        trials=2,\n",
    "        epochs=[1000],\n",
    "        accuracy=[0.8],\n",
    "        tolerance=[0.2],\n",
    "        optimizer=[\"adam\", \"sgd\"],\n",
    "        activation=[\"sigmoid\", \"relu\"],\n",
    "        dir=\"/tmp/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results is a list of (category, network-name) pairs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('adam-sigmoid', '/tmp/XOR-Test1-00001-00001'),\n",
       " ('sgd-sigmoid', '/tmp/XOR-Test1-00001-00002'),\n",
       " ('adam-relu', '/tmp/XOR-Test1-00001-00003'),\n",
       " ('sgd-relu', '/tmp/XOR-Test1-00001-00004'),\n",
       " ('adam-sigmoid', '/tmp/XOR-Test1-00002-00001'),\n",
       " ('sgd-sigmoid', '/tmp/XOR-Test1-00002-00002'),\n",
       " ('adam-relu', '/tmp/XOR-Test1-00002-00003'),\n",
       " ('sgd-relu', '/tmp/XOR-Test1-00002-00004')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exp.results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often, you may wish to plot the results of learning. This may take some time, as the function will re-load each network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"288pt\" version=\"1.1\" viewBox=\"0 0 432 288\" width=\"432pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 288  L 432 288  L 432 0  L 0 0  z \" style=\"fill:#ffffff;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 54 252  L 388.8 252  L 388.8 34.56  L 54 34.56  z \" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0  L 0 3.5  \" id=\"mc1276ca622\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"69.218182\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625  Q 24.171875 66.40625 20.328125 58.90625  Q 16.5 51.421875 16.5 36.375  Q 16.5 21.390625 20.328125 13.890625  Q 24.171875 6.390625 31.78125 6.390625  Q 39.453125 6.390625 43.28125 13.890625  Q 47.125 21.390625 47.125 36.375  Q 47.125 51.421875 43.28125 58.90625  Q 39.453125 66.40625 31.78125 66.40625  z M 31.78125 74.21875  Q 44.046875 74.21875 50.515625 64.515625  Q 56.984375 54.828125 56.984375 36.375  Q 56.984375 17.96875 50.515625 8.265625  Q 44.046875 -1.421875 31.78125 -1.421875  Q 19.53125 -1.421875 13.0625 8.265625  Q 6.59375 17.96875 6.59375 36.375  Q 6.59375 54.828125 13.0625 64.515625  Q 19.53125 74.21875 31.78125 74.21875  z \" id=\"DejaVuSans-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(66.036932 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"130.090909\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 200 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875  L 53.609375 8.296875  L 53.609375 0  L 7.328125 0  L 7.328125 8.296875  Q 12.9375 14.109375 22.625 23.890625  Q 32.328125 33.6875 34.8125 36.53125  Q 39.546875 41.84375 41.421875 45.53125  Q 43.3125 49.21875 43.3125 52.78125  Q 43.3125 58.59375 39.234375 62.25  Q 35.15625 65.921875 28.609375 65.921875  Q 23.96875 65.921875 18.8125 64.3125  Q 13.671875 62.703125 7.8125 59.421875  L 7.8125 69.390625  Q 13.765625 71.78125 18.9375 73  Q 24.125 74.21875 28.421875 74.21875  Q 39.75 74.21875 46.484375 68.546875  Q 53.21875 62.890625 53.21875 53.421875  Q 53.21875 48.921875 51.53125 44.890625  Q 49.859375 40.875 45.40625 35.40625  Q 44.1875 33.984375 37.640625 27.21875  Q 31.109375 20.453125 19.1875 8.296875  z \" id=\"DejaVuSans-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(120.547159 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"190.963636\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 400 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125  L 12.890625 25.390625  L 37.796875 25.390625  z M 35.203125 72.90625  L 47.609375 72.90625  L 47.609375 25.390625  L 58.015625 25.390625  L 58.015625 17.1875  L 47.609375 17.1875  L 47.609375 0  L 37.796875 0  L 37.796875 17.1875  L 4.890625 17.1875  L 4.890625 26.703125  z \" id=\"DejaVuSans-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(181.419886 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"251.836364\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 600 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375  Q 26.375 40.375 22.484375 35.828125  Q 18.609375 31.296875 18.609375 23.390625  Q 18.609375 15.53125 22.484375 10.953125  Q 26.375 6.390625 33.015625 6.390625  Q 39.65625 6.390625 43.53125 10.953125  Q 47.40625 15.53125 47.40625 23.390625  Q 47.40625 31.296875 43.53125 35.828125  Q 39.65625 40.375 33.015625 40.375  z M 52.59375 71.296875  L 52.59375 62.3125  Q 48.875 64.0625 45.09375 64.984375  Q 41.3125 65.921875 37.59375 65.921875  Q 27.828125 65.921875 22.671875 59.328125  Q 17.53125 52.734375 16.796875 39.40625  Q 19.671875 43.65625 24.015625 45.921875  Q 28.375 48.1875 33.59375 48.1875  Q 44.578125 48.1875 50.953125 41.515625  Q 57.328125 34.859375 57.328125 23.390625  Q 57.328125 12.15625 50.6875 5.359375  Q 44.046875 -1.421875 33.015625 -1.421875  Q 20.359375 -1.421875 13.671875 8.265625  Q 6.984375 17.96875 6.984375 36.375  Q 6.984375 53.65625 15.1875 63.9375  Q 23.390625 74.21875 37.203125 74.21875  Q 40.921875 74.21875 44.703125 73.484375  Q 48.484375 72.75 52.59375 71.296875  z \" id=\"DejaVuSans-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(242.292614 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"312.709091\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 800 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625  Q 24.75 34.625 20.71875 30.859375  Q 16.703125 27.09375 16.703125 20.515625  Q 16.703125 13.921875 20.71875 10.15625  Q 24.75 6.390625 31.78125 6.390625  Q 38.8125 6.390625 42.859375 10.171875  Q 46.921875 13.96875 46.921875 20.515625  Q 46.921875 27.09375 42.890625 30.859375  Q 38.875 34.625 31.78125 34.625  z M 21.921875 38.8125  Q 15.578125 40.375 12.03125 44.71875  Q 8.5 49.078125 8.5 55.328125  Q 8.5 64.0625 14.71875 69.140625  Q 20.953125 74.21875 31.78125 74.21875  Q 42.671875 74.21875 48.875 69.140625  Q 55.078125 64.0625 55.078125 55.328125  Q 55.078125 49.078125 51.53125 44.71875  Q 48 40.375 41.703125 38.8125  Q 48.828125 37.15625 52.796875 32.3125  Q 56.78125 27.484375 56.78125 20.515625  Q 56.78125 9.90625 50.3125 4.234375  Q 43.84375 -1.421875 31.78125 -1.421875  Q 19.734375 -1.421875 13.25 4.234375  Q 6.78125 9.90625 6.78125 20.515625  Q 6.78125 27.484375 10.78125 32.3125  Q 14.796875 37.15625 21.921875 38.8125  z M 18.3125 54.390625  Q 18.3125 48.734375 21.84375 45.5625  Q 25.390625 42.390625 31.78125 42.390625  Q 38.140625 42.390625 41.71875 45.5625  Q 45.3125 48.734375 45.3125 54.390625  Q 45.3125 60.0625 41.71875 63.234375  Q 38.140625 66.40625 31.78125 66.40625  Q 25.390625 66.40625 21.84375 63.234375  Q 18.3125 60.0625 18.3125 54.390625  z \" id=\"DejaVuSans-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(303.165341 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"373.581818\" xlink:href=\"#mc1276ca622\" y=\"252\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 1000 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875  L 28.515625 8.296875  L 28.515625 63.921875  L 10.984375 60.40625  L 10.984375 69.390625  L 28.421875 72.90625  L 38.28125 72.90625  L 38.28125 8.296875  L 54.390625 8.296875  L 54.390625 0  L 12.40625 0  z \" id=\"DejaVuSans-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(360.856818 266.598437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"190.869141\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- Epoch -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.8125 72.90625  L 55.90625 72.90625  L 55.90625 64.59375  L 19.671875 64.59375  L 19.671875 43.015625  L 54.390625 43.015625  L 54.390625 34.71875  L 19.671875 34.71875  L 19.671875 8.296875  L 56.78125 8.296875  L 56.78125 0  L 9.8125 0  z \" id=\"DejaVuSans-45\"/>\n",
       "      <path d=\"M 18.109375 8.203125  L 18.109375 -20.796875  L 9.078125 -20.796875  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.390625  Q 20.953125 51.265625 25.265625 53.625  Q 29.59375 56 35.59375 56  Q 45.5625 56 51.78125 48.09375  Q 58.015625 40.1875 58.015625 27.296875  Q 58.015625 14.40625 51.78125 6.484375  Q 45.5625 -1.421875 35.59375 -1.421875  Q 29.59375 -1.421875 25.265625 0.953125  Q 20.953125 3.328125 18.109375 8.203125  z M 48.6875 27.296875  Q 48.6875 37.203125 44.609375 42.84375  Q 40.53125 48.484375 33.40625 48.484375  Q 26.265625 48.484375 22.1875 42.84375  Q 18.109375 37.203125 18.109375 27.296875  Q 18.109375 17.390625 22.1875 11.75  Q 26.265625 6.109375 33.40625 6.109375  Q 40.53125 6.109375 44.609375 11.75  Q 48.6875 17.390625 48.6875 27.296875  z \" id=\"DejaVuSans-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625  Q 23.390625 48.390625 19.1875 42.75  Q 14.984375 37.109375 14.984375 27.296875  Q 14.984375 17.484375 19.15625 11.84375  Q 23.34375 6.203125 30.609375 6.203125  Q 37.796875 6.203125 41.984375 11.859375  Q 46.1875 17.53125 46.1875 27.296875  Q 46.1875 37.015625 41.984375 42.703125  Q 37.796875 48.390625 30.609375 48.390625  z M 30.609375 56  Q 42.328125 56 49.015625 48.375  Q 55.71875 40.765625 55.71875 27.296875  Q 55.71875 13.875 49.015625 6.21875  Q 42.328125 -1.421875 30.609375 -1.421875  Q 18.84375 -1.421875 12.171875 6.21875  Q 5.515625 13.875 5.515625 27.296875  Q 5.515625 40.765625 12.171875 48.375  Q 18.84375 56 30.609375 56  z \" id=\"DejaVuSans-6f\"/>\n",
       "      <path d=\"M 48.78125 52.59375  L 48.78125 44.1875  Q 44.96875 46.296875 41.140625 47.34375  Q 37.3125 48.390625 33.40625 48.390625  Q 24.65625 48.390625 19.8125 42.84375  Q 14.984375 37.3125 14.984375 27.296875  Q 14.984375 17.28125 19.8125 11.734375  Q 24.65625 6.203125 33.40625 6.203125  Q 37.3125 6.203125 41.140625 7.25  Q 44.96875 8.296875 48.78125 10.40625  L 48.78125 2.09375  Q 45.015625 0.34375 40.984375 -0.53125  Q 36.96875 -1.421875 32.421875 -1.421875  Q 20.0625 -1.421875 12.78125 6.34375  Q 5.515625 14.109375 5.515625 27.296875  Q 5.515625 40.671875 12.859375 48.328125  Q 20.21875 56 33.015625 56  Q 37.15625 56 41.109375 55.140625  Q 45.0625 54.296875 48.78125 52.59375  z \" id=\"DejaVuSans-63\"/>\n",
       "      <path d=\"M 54.890625 33.015625  L 54.890625 0  L 45.90625 0  L 45.90625 32.71875  Q 45.90625 40.484375 42.875 44.328125  Q 39.84375 48.1875 33.796875 48.1875  Q 26.515625 48.1875 22.3125 43.546875  Q 18.109375 38.921875 18.109375 30.90625  L 18.109375 0  L 9.078125 0  L 9.078125 75.984375  L 18.109375 75.984375  L 18.109375 46.1875  Q 21.34375 51.125 25.703125 53.5625  Q 30.078125 56 35.796875 56  Q 45.21875 56 50.046875 50.171875  Q 54.890625 44.34375 54.890625 33.015625  z \" id=\"DejaVuSans-68\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(206.089063 280.276563)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-45\"/>\n",
       "      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0  L -3.5 0  \" id=\"m6996bb08f3\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"244.795984\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.6875 12.40625  L 21 12.40625  L 21 0  L 10.6875 0  z \" id=\"DejaVuSans-2e\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(31.096875 248.595203)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"212.571772\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.2 -->\n",
       "      <g transform=\"translate(31.096875 216.370991)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"180.34756\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.4 -->\n",
       "      <g transform=\"translate(31.096875 184.146779)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"148.123348\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.6 -->\n",
       "      <g transform=\"translate(31.096875 151.922567)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"115.899136\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.8 -->\n",
       "      <g transform=\"translate(31.096875 119.698354)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"83.674924\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(31.096875 87.474142)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54\" xlink:href=\"#m6996bb08f3\" y=\"51.450711\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 1.2 -->\n",
       "      <g transform=\"translate(31.096875 55.24993)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_14\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 191.40673  L 69.826909 191.475059  L 86.262545 195.051076  L 100.567636 197.929264  L 113.046545 200.228011  L 125.829818 202.36236  L 138.613091 204.269239  L 151.700727 205.995568  L 166.005818 207.650183  L 181.832727 209.252943  L 199.790182 210.843961  L 219.269455 212.343939  L 240.270545 213.736547  L 263.097818 215.025955  L 288.36 216.226792  L 317.274545 217.371018  L 351.667636 218.500222  L 373.581818 219.130062  L 373.581818 219.130062  \" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_15\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 206.928026  L 71.348727 207.03399  L 89.610545 207.969497  L 110.611636 208.814639  L 134.960727 209.565033  L 164.179636 210.234396  L 200.094545 210.824586  L 246.053455 211.344881  L 307.534909 211.804699  L 373.581818 212.13782  L 373.581818 212.13782  \" style=\"fill:none;stroke:#008000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_16\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 211.779202  L 69.522545 211.779202  L 70.435636 212.223525  L 77.131636 215.652463  L 82.914545 218.371364  L 88.088727 220.565874  L 93.262909 222.514951  L 98.132727 224.112617  L 103.002545 225.486782  L 108.176727 226.713104  L 113.655273 227.779532  L 119.742545 228.734179  L 127.047273 229.647167  L 137.395636 230.694617  L 162.049091 232.890421  L 197.964 235.929404  L 214.704 237.147379  L 250.314545 239.559829  L 279.229091 241.295566  L 295.056 242.116364  L 295.056 242.116364  \" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_17\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 44.443636  L 69.522545 44.443636  L 70.131273 102.991935  L 70.74 140.28215  L 71.348727 164.81234  L 71.957455 181.361518  L 72.566182 192.550908  L 73.479273 199.619938  L 74.392364 204.74045  L 75.305455 208.590499  L 76.218545 211.593582  L 77.436 214.731426  L 78.653455 217.223601  L 80.175273 219.763211  L 81.088364 220.955138  L 82.001455 222.039176  L 83.523273 223.659379  L 84.132 224.182891  L 85.653818 225.455927  L 89.001818 227.795696  L 90.219273 228.518249  L 93.871636 230.495353  L 94.784727 230.907478  L 98.437091 232.522813  L 108.176727 235.829773  L 109.698545 236.234203  L 111.829091 236.794213  L 115.481455 237.643469  L 121.568727 238.852017  L 135.569455 240.874997  L 136.786909 241.012224  L 136.786909 241.012224  \" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_18\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 187.356576  L 69.826909 187.422384  L 85.653818 190.729357  L 100.872 193.660014  L 116.090182 196.357689  L 130.395273 198.678457  L 145.004727 200.825108  L 159.614182 202.74559  L 174.528 204.483624  L 190.354909 206.1011  L 207.094909 207.584687  L 225.052364 208.948715  L 244.227273 210.180184  L 265.228364 211.304685  L 289.273091 212.363278  L 316.970182 213.355516  L 350.450182 214.323853  L 373.581818 214.891844  L 373.581818 214.891844  \" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 183.505225  L 69.826909 183.601304  L 84.132 187.999626  L 95.697818 191.328394  L 106.350545 194.162636  L 116.394545 196.606215  L 126.438545 198.817325  L 136.482545 200.796302  L 146.526545 202.551723  L 157.179273 204.185198  L 168.440727 205.679527  L 180.310909 207.02538  L 193.094182 208.247282  L 207.094909 209.35737  L 222.617455 210.359269  L 239.966182 211.25199  L 259.749818 212.044164  L 282.881455 212.743227  L 310.578545 213.350981  L 344.667273 213.868111  L 373.581818 214.174366  L 373.581818 214.174366  \" style=\"fill:none;stroke:#008000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_20\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 204.515719  L 373.581818 204.515719  L 373.581818 204.515719  \" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path clip-path=\"url(#p20d7ddb3d0)\" d=\"M 69.218182 210.22152  L 69.826909 210.329294  L 74.088 211.705857  L 78.957818 213.044015  L 84.740727 214.398132  L 91.436727 215.728523  L 99.045818 217.007958  L 107.872364 218.262678  L 118.525091 219.54199  L 131.308364 220.845926  L 147.744 222.286283  L 169.049455 223.917996  L 185.180727 224.972053  L 219.269455 226.979621  L 264.924 229.721035  L 307.839273 232.537647  L 373.581818 236.974949  L 373.581818 236.974949  \" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 54 252  L 54 34.56  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 388.8 252  L 388.8 34.56  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 54 252  L 388.8 252  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 54 34.56  L 388.8 34.56  \" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"text_15\">\n",
       "    <!-- XOR-Test1 -->\n",
       "    <defs>\n",
       "     <path d=\"M 6.296875 72.90625  L 16.890625 72.90625  L 35.015625 45.796875  L 53.21875 72.90625  L 63.8125 72.90625  L 40.375 37.890625  L 65.375 0  L 54.78125 0  L 34.28125 31  L 13.625 0  L 2.984375 0  L 29 38.921875  z \" id=\"DejaVuSans-58\"/>\n",
       "     <path d=\"M 39.40625 66.21875  Q 28.65625 66.21875 22.328125 58.203125  Q 16.015625 50.203125 16.015625 36.375  Q 16.015625 22.609375 22.328125 14.59375  Q 28.65625 6.59375 39.40625 6.59375  Q 50.140625 6.59375 56.421875 14.59375  Q 62.703125 22.609375 62.703125 36.375  Q 62.703125 50.203125 56.421875 58.203125  Q 50.140625 66.21875 39.40625 66.21875  z M 39.40625 74.21875  Q 54.734375 74.21875 63.90625 63.9375  Q 73.09375 53.65625 73.09375 36.375  Q 73.09375 19.140625 63.90625 8.859375  Q 54.734375 -1.421875 39.40625 -1.421875  Q 24.03125 -1.421875 14.8125 8.828125  Q 5.609375 19.09375 5.609375 36.375  Q 5.609375 53.65625 14.8125 63.9375  Q 24.03125 74.21875 39.40625 74.21875  z \" id=\"DejaVuSans-4f\"/>\n",
       "     <path d=\"M 44.390625 34.1875  Q 47.5625 33.109375 50.5625 29.59375  Q 53.5625 26.078125 56.59375 19.921875  L 66.609375 0  L 56 0  L 46.6875 18.703125  Q 43.0625 26.03125 39.671875 28.421875  Q 36.28125 30.8125 30.421875 30.8125  L 19.671875 30.8125  L 19.671875 0  L 9.8125 0  L 9.8125 72.90625  L 32.078125 72.90625  Q 44.578125 72.90625 50.734375 67.671875  Q 56.890625 62.453125 56.890625 51.90625  Q 56.890625 45.015625 53.6875 40.46875  Q 50.484375 35.9375 44.390625 34.1875  z M 19.671875 64.796875  L 19.671875 38.921875  L 32.078125 38.921875  Q 39.203125 38.921875 42.84375 42.21875  Q 46.484375 45.515625 46.484375 51.90625  Q 46.484375 58.296875 42.84375 61.546875  Q 39.203125 64.796875 32.078125 64.796875  z \" id=\"DejaVuSans-52\"/>\n",
       "     <path d=\"M 4.890625 31.390625  L 31.203125 31.390625  L 31.203125 23.390625  L 4.890625 23.390625  z \" id=\"DejaVuSans-2d\"/>\n",
       "     <path d=\"M -0.296875 72.90625  L 61.375 72.90625  L 61.375 64.59375  L 35.5 64.59375  L 35.5 0  L 25.59375 0  L 25.59375 64.59375  L -0.296875 64.59375  z \" id=\"DejaVuSans-54\"/>\n",
       "     <path d=\"M 56.203125 29.59375  L 56.203125 25.203125  L 14.890625 25.203125  Q 15.484375 15.921875 20.484375 11.0625  Q 25.484375 6.203125 34.421875 6.203125  Q 39.59375 6.203125 44.453125 7.46875  Q 49.3125 8.734375 54.109375 11.28125  L 54.109375 2.78125  Q 49.265625 0.734375 44.1875 -0.34375  Q 39.109375 -1.421875 33.890625 -1.421875  Q 20.796875 -1.421875 13.15625 6.1875  Q 5.515625 13.8125 5.515625 26.8125  Q 5.515625 40.234375 12.765625 48.109375  Q 20.015625 56 32.328125 56  Q 43.359375 56 49.78125 48.890625  Q 56.203125 41.796875 56.203125 29.59375  z M 47.21875 32.234375  Q 47.125 39.59375 43.09375 43.984375  Q 39.0625 48.390625 32.421875 48.390625  Q 24.90625 48.390625 20.390625 44.140625  Q 15.875 39.890625 15.1875 32.171875  z \" id=\"DejaVuSans-65\"/>\n",
       "     <path d=\"M 44.28125 53.078125  L 44.28125 44.578125  Q 40.484375 46.53125 36.375 47.5  Q 32.28125 48.484375 27.875 48.484375  Q 21.1875 48.484375 17.84375 46.4375  Q 14.5 44.390625 14.5 40.28125  Q 14.5 37.15625 16.890625 35.375  Q 19.28125 33.59375 26.515625 31.984375  L 29.59375 31.296875  Q 39.15625 29.25 43.1875 25.515625  Q 47.21875 21.78125 47.21875 15.09375  Q 47.21875 7.46875 41.1875 3.015625  Q 35.15625 -1.421875 24.609375 -1.421875  Q 20.21875 -1.421875 15.453125 -0.5625  Q 10.6875 0.296875 5.421875 2  L 5.421875 11.28125  Q 10.40625 8.6875 15.234375 7.390625  Q 20.0625 6.109375 24.8125 6.109375  Q 31.15625 6.109375 34.5625 8.28125  Q 37.984375 10.453125 37.984375 14.40625  Q 37.984375 18.0625 35.515625 20.015625  Q 33.0625 21.96875 24.703125 23.78125  L 21.578125 24.515625  Q 13.234375 26.265625 9.515625 29.90625  Q 5.8125 33.546875 5.8125 39.890625  Q 5.8125 47.609375 11.28125 51.796875  Q 16.75 56 26.8125 56  Q 31.78125 56 36.171875 55.265625  Q 40.578125 54.546875 44.28125 53.078125  z \" id=\"DejaVuSans-73\"/>\n",
       "     <path d=\"M 18.3125 70.21875  L 18.3125 54.6875  L 36.8125 54.6875  L 36.8125 47.703125  L 18.3125 47.703125  L 18.3125 18.015625  Q 18.3125 11.328125 20.140625 9.421875  Q 21.96875 7.515625 27.59375 7.515625  L 36.8125 7.515625  L 36.8125 0  L 27.59375 0  Q 17.1875 0 13.234375 3.875  Q 9.28125 7.765625 9.28125 18.015625  L 9.28125 47.703125  L 2.6875 47.703125  L 2.6875 54.6875  L 9.28125 54.6875  L 9.28125 70.21875  z \" id=\"DejaVuSans-74\"/>\n",
       "    </defs>\n",
       "    <g transform=\"translate(189.615 28.56)scale(0.12 -0.12)\">\n",
       "     <use xlink:href=\"#DejaVuSans-58\"/>\n",
       "     <use x=\"68.412109\" xlink:href=\"#DejaVuSans-4f\"/>\n",
       "     <use x=\"147.123047\" xlink:href=\"#DejaVuSans-52\"/>\n",
       "     <use x=\"216.542969\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "     <use x=\"252.486328\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "     <use x=\"313.304688\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "     <use x=\"374.828125\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     <use x=\"426.927734\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     <use x=\"466.136719\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 278.525 159.985  L 381.8 159.985  Q 383.8 159.985 383.8 157.985  L 383.8 41.56  Q 383.8 39.56 381.8 39.56  L 278.525 39.56  Q 276.525 39.56 276.525 41.56  L 276.525 157.985  Q 276.525 159.985 278.525 159.985  z \" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_22\">\n",
       "     <path d=\"M 280.525 47.658437  L 300.525 47.658437  \" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_23\"/>\n",
       "    <g id=\"text_16\">\n",
       "     <!-- adam-sigmoid -->\n",
       "     <defs>\n",
       "      <path d=\"M 34.28125 27.484375  Q 23.390625 27.484375 19.1875 25  Q 14.984375 22.515625 14.984375 16.5  Q 14.984375 11.71875 18.140625 8.90625  Q 21.296875 6.109375 26.703125 6.109375  Q 34.1875 6.109375 38.703125 11.40625  Q 43.21875 16.703125 43.21875 25.484375  L 43.21875 27.484375  z M 52.203125 31.203125  L 52.203125 0  L 43.21875 0  L 43.21875 8.296875  Q 40.140625 3.328125 35.546875 0.953125  Q 30.953125 -1.421875 24.3125 -1.421875  Q 15.921875 -1.421875 10.953125 3.296875  Q 6 8.015625 6 15.921875  Q 6 25.140625 12.171875 29.828125  Q 18.359375 34.515625 30.609375 34.515625  L 43.21875 34.515625  L 43.21875 35.40625  Q 43.21875 41.609375 39.140625 45  Q 35.0625 48.390625 27.6875 48.390625  Q 23 48.390625 18.546875 47.265625  Q 14.109375 46.140625 10.015625 43.890625  L 10.015625 52.203125  Q 14.9375 54.109375 19.578125 55.046875  Q 24.21875 56 28.609375 56  Q 40.484375 56 46.34375 49.84375  Q 52.203125 43.703125 52.203125 31.203125  z \" id=\"DejaVuSans-61\"/>\n",
       "      <path d=\"M 45.40625 46.390625  L 45.40625 75.984375  L 54.390625 75.984375  L 54.390625 0  L 45.40625 0  L 45.40625 8.203125  Q 42.578125 3.328125 38.25 0.953125  Q 33.9375 -1.421875 27.875 -1.421875  Q 17.96875 -1.421875 11.734375 6.484375  Q 5.515625 14.40625 5.515625 27.296875  Q 5.515625 40.1875 11.734375 48.09375  Q 17.96875 56 27.875 56  Q 33.9375 56 38.25 53.625  Q 42.578125 51.265625 45.40625 46.390625  z M 14.796875 27.296875  Q 14.796875 17.390625 18.875 11.75  Q 22.953125 6.109375 30.078125 6.109375  Q 37.203125 6.109375 41.296875 11.75  Q 45.40625 17.390625 45.40625 27.296875  Q 45.40625 37.203125 41.296875 42.84375  Q 37.203125 48.484375 30.078125 48.484375  Q 22.953125 48.484375 18.875 42.84375  Q 14.796875 37.203125 14.796875 27.296875  z \" id=\"DejaVuSans-64\"/>\n",
       "      <path d=\"M 52 44.1875  Q 55.375 50.25 60.0625 53.125  Q 64.75 56 71.09375 56  Q 79.640625 56 84.28125 50.015625  Q 88.921875 44.046875 88.921875 33.015625  L 88.921875 0  L 79.890625 0  L 79.890625 32.71875  Q 79.890625 40.578125 77.09375 44.375  Q 74.3125 48.1875 68.609375 48.1875  Q 61.625 48.1875 57.5625 43.546875  Q 53.515625 38.921875 53.515625 30.90625  L 53.515625 0  L 44.484375 0  L 44.484375 32.71875  Q 44.484375 40.625 41.703125 44.40625  Q 38.921875 48.1875 33.109375 48.1875  Q 26.21875 48.1875 22.15625 43.53125  Q 18.109375 38.875 18.109375 30.90625  L 18.109375 0  L 9.078125 0  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.1875  Q 21.1875 51.21875 25.484375 53.609375  Q 29.78125 56 35.6875 56  Q 41.65625 56 45.828125 52.96875  Q 50 49.953125 52 44.1875  z \" id=\"DejaVuSans-6d\"/>\n",
       "      <path d=\"M 9.421875 54.6875  L 18.40625 54.6875  L 18.40625 0  L 9.421875 0  z M 9.421875 75.984375  L 18.40625 75.984375  L 18.40625 64.59375  L 9.421875 64.59375  z \" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 45.40625 27.984375  Q 45.40625 37.75 41.375 43.109375  Q 37.359375 48.484375 30.078125 48.484375  Q 22.859375 48.484375 18.828125 43.109375  Q 14.796875 37.75 14.796875 27.984375  Q 14.796875 18.265625 18.828125 12.890625  Q 22.859375 7.515625 30.078125 7.515625  Q 37.359375 7.515625 41.375 12.890625  Q 45.40625 18.265625 45.40625 27.984375  z M 54.390625 6.78125  Q 54.390625 -7.171875 48.1875 -13.984375  Q 42 -20.796875 29.203125 -20.796875  Q 24.46875 -20.796875 20.265625 -20.09375  Q 16.0625 -19.390625 12.109375 -17.921875  L 12.109375 -9.1875  Q 16.0625 -11.328125 19.921875 -12.34375  Q 23.78125 -13.375 27.78125 -13.375  Q 36.625 -13.375 41.015625 -8.765625  Q 45.40625 -4.15625 45.40625 5.171875  L 45.40625 9.625  Q 42.625 4.78125 38.28125 2.390625  Q 33.9375 0 27.875 0  Q 17.828125 0 11.671875 7.65625  Q 5.515625 15.328125 5.515625 27.984375  Q 5.515625 40.671875 11.671875 48.328125  Q 17.828125 56 27.875 56  Q 33.9375 56 38.28125 53.609375  Q 42.625 51.21875 45.40625 46.390625  L 45.40625 54.6875  L 54.390625 54.6875  z \" id=\"DejaVuSans-67\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(308.525 51.158437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"124.755859\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"186.035156\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"283.447266\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"319.53125\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"371.630859\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"399.414062\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"462.890625\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"560.302734\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"621.484375\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"649.267578\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 280.525 62.336563  L 300.525 62.336563  \" style=\"fill:none;stroke:#008000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\"/>\n",
       "    <g id=\"text_17\">\n",
       "     <!-- sgd-sigmoid -->\n",
       "     <g transform=\"translate(308.525 65.836563)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"115.576172\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"179.052734\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"215.136719\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"267.236328\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"295.019531\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"358.496094\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"455.908203\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"517.089844\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"544.873047\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 280.525 77.014687  L 300.525 77.014687  \" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_27\"/>\n",
       "    <g id=\"text_18\">\n",
       "     <!-- adam-relu -->\n",
       "     <defs>\n",
       "      <path d=\"M 41.109375 46.296875  Q 39.59375 47.171875 37.8125 47.578125  Q 36.03125 48 33.890625 48  Q 26.265625 48 22.1875 43.046875  Q 18.109375 38.09375 18.109375 28.8125  L 18.109375 0  L 9.078125 0  L 9.078125 54.6875  L 18.109375 54.6875  L 18.109375 46.1875  Q 20.953125 51.171875 25.484375 53.578125  Q 30.03125 56 36.53125 56  Q 37.453125 56 38.578125 55.875  Q 39.703125 55.765625 41.0625 55.515625  z \" id=\"DejaVuSans-72\"/>\n",
       "      <path d=\"M 9.421875 75.984375  L 18.40625 75.984375  L 18.40625 0  L 9.421875 0  z \" id=\"DejaVuSans-6c\"/>\n",
       "      <path d=\"M 8.5 21.578125  L 8.5 54.6875  L 17.484375 54.6875  L 17.484375 21.921875  Q 17.484375 14.15625 20.5 10.265625  Q 23.53125 6.390625 29.59375 6.390625  Q 36.859375 6.390625 41.078125 11.03125  Q 45.3125 15.671875 45.3125 23.6875  L 45.3125 54.6875  L 54.296875 54.6875  L 54.296875 0  L 45.3125 0  L 45.3125 8.40625  Q 42.046875 3.421875 37.71875 1  Q 33.40625 -1.421875 27.6875 -1.421875  Q 18.265625 -1.421875 13.375 4.4375  Q 8.5 10.296875 8.5 21.578125  z M 31.109375 56  z \" id=\"DejaVuSans-75\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(308.525 80.514687)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"124.755859\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"186.035156\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"283.447266\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"319.53125\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"360.613281\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"422.136719\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"449.919922\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_28\">\n",
       "     <path d=\"M 280.525 91.692813  L 300.525 91.692813  \" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_29\"/>\n",
       "    <g id=\"text_19\">\n",
       "     <!-- sgd-relu -->\n",
       "     <g transform=\"translate(308.525 95.192813)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"115.576172\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"179.052734\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"215.136719\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"256.21875\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"317.742188\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"345.525391\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_30\">\n",
       "     <path d=\"M 280.525 106.370937  L 300.525 106.370937  \" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_31\"/>\n",
       "    <g id=\"text_20\">\n",
       "     <!-- adam-sigmoid -->\n",
       "     <g transform=\"translate(308.525 109.870937)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"124.755859\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"186.035156\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"283.447266\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"319.53125\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"371.630859\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"399.414062\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"462.890625\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"560.302734\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"621.484375\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"649.267578\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_32\">\n",
       "     <path d=\"M 280.525 121.049062  L 300.525 121.049062  \" style=\"fill:none;stroke:#008000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_33\"/>\n",
       "    <g id=\"text_21\">\n",
       "     <!-- sgd-sigmoid -->\n",
       "     <g transform=\"translate(308.525 124.549062)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"115.576172\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"179.052734\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"215.136719\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"267.236328\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"295.019531\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"358.496094\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"455.908203\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"517.089844\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"544.873047\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_34\">\n",
       "     <path d=\"M 280.525 135.727187  L 300.525 135.727187  \" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_35\"/>\n",
       "    <g id=\"text_22\">\n",
       "     <!-- adam-relu -->\n",
       "     <g transform=\"translate(308.525 139.227187)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"124.755859\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"186.035156\" xlink:href=\"#DejaVuSans-6d\"/>\n",
       "      <use x=\"283.447266\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"319.53125\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"360.613281\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"422.136719\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"449.919922\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_36\">\n",
       "     <path d=\"M 280.525 150.405312  L 300.525 150.405312  \" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_37\"/>\n",
       "    <g id=\"text_23\">\n",
       "     <!-- sgd-relu -->\n",
       "     <g transform=\"translate(308.525 153.905312)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-67\"/>\n",
       "      <use x=\"115.576172\" xlink:href=\"#DejaVuSans-64\"/>\n",
       "      <use x=\"179.052734\" xlink:href=\"#DejaVuSans-2d\"/>\n",
       "      <use x=\"215.136719\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"256.21875\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"317.742188\" xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"345.525391\" xlink:href=\"#DejaVuSans-75\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p20d7ddb3d0\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"54\" y=\"34.56\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exp.plot(\"loss\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that each category has its own color.\n",
    "\n",
    "There is also a generic apply method for calling a function with each of the (category, network-names). Ususally, you would probably want to re-load the network, and perform some operation in the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('adam-sigmoid', '/tmp/XOR-Test1-00001-00001'),\n",
       " ('sgd-sigmoid', '/tmp/XOR-Test1-00001-00002'),\n",
       " ('adam-relu', '/tmp/XOR-Test1-00001-00003'),\n",
       " ('sgd-relu', '/tmp/XOR-Test1-00001-00004'),\n",
       " ('adam-sigmoid', '/tmp/XOR-Test1-00002-00001'),\n",
       " ('sgd-sigmoid', '/tmp/XOR-Test1-00002-00002'),\n",
       " ('adam-relu', '/tmp/XOR-Test1-00002-00003'),\n",
       " ('sgd-relu', '/tmp/XOR-Test1-00002-00004')]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exp.apply(lambda category, exp_name: (category, exp_name))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}