-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add basic Excel-oriented import-export example (#188)
- Basic example using Python. - Uses Excel (XLSX) configuration file. - Notebook is computed but the saved file is not included.
- Loading branch information
1 parent
72daca2
commit cd2776e
Showing
2 changed files
with
323 additions
and
0 deletions.
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,323 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# PoPS Border simulation using an XLSX configuration file\n", | ||
| "\n", | ||
| "## Test example \n", | ||
| "\n", | ||
| "First, we complete the Excel template `data/small_config.xlsx` with the minimal configuration parameter required to run the simulation.\n", | ||
| "\n", | ||
| "The Excel configuration should include all parameters related to:\n", | ||
| "1. [consignments](../../docs/consignments.md) (what is imported, from where, in what amounts), \n", | ||
| "2. [contaminants](../../docs/contamination.md) (proportion and distribution of contaminated consignments, boxes, and items), and \n", | ||
| "3. [inspection](../../docs/inspections.md) (sampling strategies and rates)\n", | ||
| "\n", | ||
| "There is a help file (linked above) for each of these sections that goes into greater detail about how to define the parameters." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Set up the workspace\n", | ||
| "\n", | ||
| "First, we load the needed `popsborder` functions." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 2, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# Load required functions and packages\n", | ||
| "\n", | ||
| "from popsborder.simulation import run_simulation\n", | ||
| "from popsborder.inputs import load_configuration\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We provide the location of our Excel configuration file. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 3, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "xlsx_loc = \"data/small_config.xlsx\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We use the `load_configuration` function to import the simulation settings from the file. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 4, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "base_config = load_configuration(xlsx_loc)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Run the simulation \n", | ||
| "\n", | ||
| "We use the `run_simulation` function to run the simulation. We assign the output of this function to a variable, so that we can examine the results later.\n", | ||
| "\n", | ||
| "We can include the `pretty` and `verbose` parameters below to visualize the contamination and output directly in the notebook." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 5, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "━━ Consignment ━━ Boxes: 3 ━━ Items: 60 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n", | ||
| "🐛 ✿ ✿ 🐛 ✿ 🐛 🐛 🐛 🐛 🐛 ✿ ✿ 🐛 🐛 ✿ ✿ 🐛 🐛 ✿ ✿ | ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ ✿ ✿ 🐛 ✿ ✿ 🐛 🐛 ✿ ✿ ✿ ✿ ✿ | 🐛 🐛 🐛 ✿ ✿ 🐛 ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ 🐛 ✿ ✿ ✿ ✿ ✿\n", | ||
| "Inspection worked, found contaminant [TP]\n", | ||
| "━━ Consignment ━━ Boxes: 2 ━━ Items: 40 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n", | ||
| "🐛 🐛 🐛 ✿ 🐛 🐛 🐛 ✿ 🐛 ✿ 🐛 🐛 ✿ ✿ ✿ ✿ 🐛 ✿ 🐛 🐛 | ✿ 🐛 ✿ 🐛 ✿ ✿ 🐛 🐛 ✿ ✿ ✿ ✿ 🐛 ✿ 🐛 ✿ 🐛 🐛 🐛 🐛\n", | ||
| "Inspection worked, found contaminant [TP]\n", | ||
| "━━ Consignment ━━ Boxes: 5 ━━ Items: 100 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n", | ||
| "✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ ✿ ✿ ✿ 🐛 ✿ 🐛 🐛 ✿ | 🐛 ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ | ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 🐛 ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 | ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ 🐛 ✿ ✿ 🐛 🐛 🐛 ✿ ✿ ✿ ✿ ✿ ✿ | ✿ ✿ 🐛 ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ ✿ 🐛 ✿ ✿\n", | ||
| "Inspection worked, found contaminant [TP]\n", | ||
| "━━ Consignment ━━ Boxes: 4 ━━ Items: 80 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n", | ||
| "✿ ✿ ✿ 🐛 ✿ 🐛 ✿ 🐛 ✿ 🐛 ✿ ✿ 🐛 🐛 🐛 ✿ ✿ ✿ ✿ ✿ | 🐛 ✿ ✿ ✿ ✿ ✿ 🐛 ✿ 🐛 ✿ ✿ ✿ ✿ 🐛 ✿ ✿ 🐛 ✿ ✿ ✿ | ✿ ✿ ✿ 🐛 🐛 🐛 🐛 🐛 ✿ 🐛 ✿ ✿ 🐛 🐛 ✿ ✿ ✿ 🐛 ✿ ✿ | ✿ ✿ ✿ ✿ ✿ 🐛 ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ ✿ 🐛 🐛 🐛 ✿\n", | ||
| "Inspection worked, found contaminant [TP]\n", | ||
| "Missing {missing:.0f}% of contaminated consignments.\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "result = run_simulation(\n", | ||
| " config=base_config,\n", | ||
| " seed=42,\n", | ||
| " num_simulations=1,\n", | ||
| " num_consignments=4,\n", | ||
| " pretty=\"boxes\",\n", | ||
| " verbose=True\n", | ||
| " )\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Export simulation results\n", | ||
| "\n", | ||
| "The summary of results can also be converted to a tabular format with the `pandas` library and exported to Excel or used directly for further analyses. \n", | ||
| "\n", | ||
| "First, we import an additional needed `popsborder` function and the `pandas` library." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 6, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from popsborder.outputs import save_simulation_result_to_pandas \n", | ||
| "import pandas" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We use `save_simulation_result_to_pandas` to convert our simulation results into a dataframe" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "results_pd = save_simulation_result_to_pandas(result)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The resulting dataframe can be used directly here for further analyses." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 8, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "data": { | ||
| "text/html": [ | ||
| "<div>\n", | ||
| "<style scoped>\n", | ||
| " .dataframe tbody tr th:only-of-type {\n", | ||
| " vertical-align: middle;\n", | ||
| " }\n", | ||
| "\n", | ||
| " .dataframe tbody tr th {\n", | ||
| " vertical-align: top;\n", | ||
| " }\n", | ||
| "\n", | ||
| " .dataframe thead th {\n", | ||
| " text-align: right;\n", | ||
| " }\n", | ||
| "</style>\n", | ||
| "<table border=\"1\" class=\"dataframe\">\n", | ||
| " <thead>\n", | ||
| " <tr style=\"text-align: right;\">\n", | ||
| " <th></th>\n", | ||
| " <th>missing</th>\n", | ||
| " <th>false_neg</th>\n", | ||
| " <th>missed_within_tolerance</th>\n", | ||
| " <th>intercepted</th>\n", | ||
| " <th>num_inspections</th>\n", | ||
| " <th>num_boxes</th>\n", | ||
| " <th>num_items</th>\n", | ||
| " <th>avg_boxes_opened_completion</th>\n", | ||
| " <th>avg_boxes_opened_detection</th>\n", | ||
| " <th>pct_boxes_opened_completion</th>\n", | ||
| " <th>...</th>\n", | ||
| " <th>pct_contaminant_unreported_if_detection</th>\n", | ||
| " <th>true_contamination_rate</th>\n", | ||
| " <th>max_missed_contamination_rate</th>\n", | ||
| " <th>avg_missed_contamination_rate</th>\n", | ||
| " <th>max_intercepted_contamination_rate</th>\n", | ||
| " <th>avg_intercepted_contamination_rate</th>\n", | ||
| " <th>false_negative_present</th>\n", | ||
| " <th>true_positive_present</th>\n", | ||
| " <th>total_intercepted_contaminants</th>\n", | ||
| " <th>total_missed_contaminants</th>\n", | ||
| " </tr>\n", | ||
| " </thead>\n", | ||
| " <tbody>\n", | ||
| " <tr>\n", | ||
| " <th>0</th>\n", | ||
| " <td>0.0</td>\n", | ||
| " <td>0.0</td>\n", | ||
| " <td>0.0</td>\n", | ||
| " <td>4.0</td>\n", | ||
| " <td>4.0</td>\n", | ||
| " <td>14.0</td>\n", | ||
| " <td>280.0</td>\n", | ||
| " <td>3.5</td>\n", | ||
| " <td>1.25</td>\n", | ||
| " <td>100.0</td>\n", | ||
| " <td>...</td>\n", | ||
| " <td>53.333333</td>\n", | ||
| " <td>0.345625</td>\n", | ||
| " <td>None</td>\n", | ||
| " <td>None</td>\n", | ||
| " <td>0.55</td>\n", | ||
| " <td>0.345625</td>\n", | ||
| " <td>0</td>\n", | ||
| " <td>1</td>\n", | ||
| " <td>85.0</td>\n", | ||
| " <td>0.0</td>\n", | ||
| " </tr>\n", | ||
| " </tbody>\n", | ||
| "</table>\n", | ||
| "<p>1 rows × 25 columns</p>\n", | ||
| "</div>" | ||
| ], | ||
| "text/plain": [ | ||
| " missing false_neg missed_within_tolerance intercepted num_inspections \\\n", | ||
| "0 0.0 0.0 0.0 4.0 4.0 \n", | ||
| "\n", | ||
| " num_boxes num_items avg_boxes_opened_completion \\\n", | ||
| "0 14.0 280.0 3.5 \n", | ||
| "\n", | ||
| " avg_boxes_opened_detection pct_boxes_opened_completion ... \\\n", | ||
| "0 1.25 100.0 ... \n", | ||
| "\n", | ||
| " pct_contaminant_unreported_if_detection true_contamination_rate \\\n", | ||
| "0 53.333333 0.345625 \n", | ||
| "\n", | ||
| " max_missed_contamination_rate avg_missed_contamination_rate \\\n", | ||
| "0 None None \n", | ||
| "\n", | ||
| " max_intercepted_contamination_rate avg_intercepted_contamination_rate \\\n", | ||
| "0 0.55 0.345625 \n", | ||
| "\n", | ||
| " false_negative_present true_positive_present \\\n", | ||
| "0 0 1 \n", | ||
| "\n", | ||
| " total_intercepted_contaminants total_missed_contaminants \n", | ||
| "0 85.0 0.0 \n", | ||
| "\n", | ||
| "[1 rows x 25 columns]" | ||
| ] | ||
| }, | ||
| "execution_count": 8, | ||
| "metadata": {}, | ||
| "output_type": "execute_result" | ||
| } | ||
| ], | ||
| "source": [ | ||
| "results_pd" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The dataframe can also be exported to XLSX (or other formats, like CSV) to use later." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 10, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "results_pd.to_excel('data/small_config_results.xlsx')" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "interpreter": { | ||
| "hash": "35a97950b3485d686f65df065bbd4f3ea2db4d126aaaac47be7ba4d8431a0071" | ||
| }, | ||
| "kernelspec": { | ||
| "display_name": "Python 3.9.7 ('pops_border')", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.9.7" | ||
| }, | ||
| "orig_nbformat": 4 | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |