integrations/kerastuner/showcase/Neptune-Keras-Tuner.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G4ny3mHCFrqG",
    "tags": [
     "header"
    ]
   },
   "source": [
    "# Neptune + Keras Tuner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8iu66Yz2SIWz",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Before you start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xndcBi_EUFVi",
    "tags": [
     "header",
     "installation"
    ]
   },
   "source": [
    "### Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jl90ahSMvhmV",
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet keras-tuner==1.0.2 tensorflow==2.4.1 plotly==4.14.3 neptune-client==0.4.133 neptune-contrib[monitoring]==0.26.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1As3OW3LGqKg",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Import libraries and prepare dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FufE4GTzvRZG",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "from kerastuner.tuners import  BayesianOptimization\n",
    "\n",
    "(x, y), (val_x, val_y) = keras.datasets.mnist.load_data()\n",
    "x = x.astype('float32') / 255.\n",
    "val_x = val_x.astype('float32') / 255.\n",
    "\n",
    "x = x[:10000]\n",
    "y = y[:10000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K-FNh8EKV40p",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Create a model building function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RHuuJb5VV94w",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "def build_model(hp):\n",
    "    model = keras.Sequential()\n",
    "    model.add(layers.Flatten(input_shape=(28, 28)))\n",
    "    for i in range(hp.Int('num_layers', 2, 20)):\n",
    "        model.add(layers.Dense(units=hp.Int('units_' + str(i), 32, 512, 32),\n",
    "                               activation='relu'))\n",
    "    model.add(layers.Dense(10, activation='softmax'))\n",
    "    model.compile(\n",
    "        optimizer=keras.optimizers.Adam(\n",
    "            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),\n",
    "        loss='sparse_categorical_crossentropy',\n",
    "        metrics=['accuracy'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "If9d2A8zUynE",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune\n",
    "\n",
    "neptune.init(api_token='ANONYMOUS', project_qualified_name='shared/keras-tuner-integration')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "auuzaMR3Sei2",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Neptune gives you an option of logging data under a public folder as an anonymous user. This is great when you are just trying out the application and don't have a Neptune account yet.  \n",
    "\n",
    "---\n",
    "\n",
    "If you already have a [Neptune account](https://neptune.ai/register), you can create your own experiment and start logging to it using your personal API token. Pass your `username` to the `project_qualified_name` argument of the `neptune.init()` method: `project_qualified_name='YOUR_USERNAME/YOUR_PROJECT_NAME`. If you don't have a project yet, keep `/sandbox` at the end. The `sandbox` project is automatically created for you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "X9i-HAeJDLO-",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Quickstart"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CHp9zfqBToiX",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 1: Create an Experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Be3xPNTHTsIz",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "This creates an experiment in Neptune.\n",
    "\n",
    "Once you have a live experiment you can log things to it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "t5VBefIDwBjR",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.create_experiment('bayesian-sweep')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_QdNmNLIXYXx",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Click on the link above to open this experiment in Neptune.\n",
    "\n",
    "For now it is empty but keep the tab with experiment open to see what happens next. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XomuTzlsVILc",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 2: Pass Neptune Logger to Tuner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "82879lQuVfsJ",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptunecontrib.monitoring.kerastuner as npt_utils \n",
    "\n",
    "tuner =  BayesianOptimization(\n",
    "    build_model,\n",
    "    objective='val_accuracy',\n",
    "    max_trials=10,\n",
    "    num_initial_points=3,\n",
    "    executions_per_trial=3,\n",
    "    project_name='bayesian-sweep',\n",
    "    logger=npt_utils.NeptuneLogger())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xbkNjVVLVl8t",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 3: Run the search and monitor it in Neptune "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can view the logging live in the Neptune tab once you run the below cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "40McUYc5wrva",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "tuner.search(x=x,\n",
    "             y=y,\n",
    "             epochs=5,\n",
    "             validation_data=(val_x, val_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 4: Log additional sweep information after the sweep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Log information like best parameters, best score, search space, and project dir to Neptune. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "npt_utils.log_tuner_info(tuner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P04k_nFiXl63",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 5: Stop logging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "When you track experiments with Neptune in Jupyter notebooks you need to explicitly stop the experiment by running ```neptune.stop()```.\n",
    "\n",
    "If you are running Neptune in regular ```.py``` scripts it will stop automatically when your code stops running."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "T5teGbiLwuEi",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Tcq00KslIdDd",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Explore results in the Neptune UI"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Neptune-Keras-Tuner.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}