oemof_tsam.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b9d1a9ca-447b-4764-86a8-1308998a2b13",
   "metadata": {},
   "source": [
    "# Tutorial: Using tsam with oemof.tabular\n",
    "\n",
    "@oemof anniversary meeting 2024\n",
    "\n",
    "by srhbrnds\n",
    "\n",
    "## Useful literature\n",
    "\n",
    "- https://energyinformatics.springeropen.com/articles/10.1186/s42162-022-00208-5 , (Blanke, et. al.: **Time series aggregation for energy system design: review and extension of modelling seasonal storages**, 2022)\n",
    "\n",
    "- https://www.mdpi.com/1996-1073/13/3/641 (Kotzur, et.al: **A Review on Time Series Aggregation Methods for Energy System Models**, 2020).\n",
    "\n",
    "- https://www.sciencedirect.com/science/article/abs/pii/S0306261922004342 (Hoffmann, et. al: **The Pareto-Optimal Temporal Aggregation\n",
    "of Energy System Models**, 2022)\n",
    "\n",
    "- **oemof-solph example**: https://github.com/oemof/oemof-solph/tree/feature/integrate_tsam/examples/tsam\n",
    "\n",
    "- **oemof-tabular example**: https://github.com/oemof/oemof-tabular/tree/features/add-tsam-to-datapackage/src/oemof/tabular/examples/datapackages/dispatch_tsam_without_multi_periods\n",
    "\n",
    "## Limitations\n",
    "- segmentation doesn't work so far\n",
    "- works currently only on experimental branches in oemof.solph and oemof.tabular\n",
    "- python versions 3.9, 3.10\n",
    "\n",
    "## How it works"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3ec5fe5b-ead0-4bb1-98bc-918ad1aef868",
   "metadata": {},
   "source": [
    "### oemof.tabular example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9a3e410-ae32-4cb5-9387-5ea0714632c8",
   "metadata": {},
   "source": [
    "#### Make imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "51060811-dd51-4522-bf90-76d197a5f637",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "\n",
    "import pandas as pd\n",
    "import tsam.timeseriesaggregation as tsam\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "import oemof_tsam_helpers as helpers\n",
    "\n",
    "from oemof.solph import EnergySystem, Model, processing\n",
    "\n",
    "# DONT REMOVE THIS LINE!\n",
    "from oemof.tabular import datapackage  # noqa\n",
    "from oemof.tabular.constraint_facades import CONSTRAINT_TYPE_MAP\n",
    "from oemof.tabular.facades import TYPEMAP\n",
    "from oemof.tabular.postprocessing import calculations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8458da2d-c07e-49ed-92df-6effdcf3cc14",
   "metadata": {},
   "source": [
    "#### Set scenario names for origin and target (tsam)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "04afea48-4ba7-428c-8946-2158dedb6de6",
   "metadata": {},
   "outputs": [],
   "source": [
    "datapackage_name_origin =\"dispatch\"\n",
    "datapackage_name_tsam= \"dispatch_tsam\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1db99516-8ab0-4692-886f-23be0af2e8ce",
   "metadata": {},
   "source": [
    "#### Specify paths for datapackage origin and target (tsam)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "76fac4c6-06d2-42fe-ba65-cc59dcaaabba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Origin path: /home/sarah/git_repos/oemof_tsam_tutorial/dispatch\n",
      "Origin path exists?: True\n",
      "Target path: /home/sarah/git_repos/oemof_tsam_tutorial/dispatch_tsam\n",
      "Target path exists?: True\n"
     ]
    }
   ],
   "source": [
    "# specify paths to datapackage\n",
    "datapackage_path_origin=helpers.DATAPACKAGE_DIR / datapackage_name_origin\n",
    "datapackage_path_tsam=helpers.DATAPACKAGE_DIR  / datapackage_name_tsam\n",
    "\n",
    "print('Origin path:', datapackage_path_origin)\n",
    "print('Origin path exists?:', datapackage_path_origin.exists())\n",
    "\n",
    "print('Target path:', datapackage_path_tsam)\n",
    "print('Target path exists?:', datapackage_path_tsam.exists())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88e08ff0-f7f3-429b-9b18-03c1210af214",
   "metadata": {},
   "source": [
    "#### Prepare time series data\n",
    "\n",
    "All timeseries data needs to be stored in one single DataFrame. For defaults check the oemof_tsam_helpers.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "fc556382-0643-42b5-a720-ef6a668b5f70",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>electricity-load-profile</th>\n",
       "      <th>pv-profile</th>\n",
       "      <th>wind-profile</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>timeindex</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2050-01-01T00:00:00Z</th>\n",
       "      <td>0.000075</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.147532</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2050-01-01T01:00:00Z</th>\n",
       "      <td>0.000071</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.184181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2050-01-01T02:00:00Z</th>\n",
       "      <td>0.000069</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.223937</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2050-01-01T03:00:00Z</th>\n",
       "      <td>0.000067</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.255732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2050-01-01T04:00:00Z</th>\n",
       "      <td>0.000066</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.268440</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      electricity-load-profile  pv-profile  wind-profile\n",
       "timeindex                                                               \n",
       "2050-01-01T00:00:00Z                  0.000075         0.0      0.147532\n",
       "2050-01-01T01:00:00Z                  0.000071         0.0      0.184181\n",
       "2050-01-01T02:00:00Z                  0.000069         0.0      0.223937\n",
       "2050-01-01T03:00:00Z                  0.000067         0.0      0.255732\n",
       "2050-01-01T04:00:00Z                  0.000066         0.0      0.268440"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"Crawl the sequences csv-files of oemof-tabular data/sequences path and merges them into one single DataFrame.\n",
    "    Parameters\n",
    "    ----------\n",
    "    path : Path\n",
    "        The path object pointing to the datapackage JSON-file.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    profiles : pd.DataFrame\n",
    "        DataFrame that contains all sequence data specified in the\n",
    "        oemof-tabular datapackage.\n",
    "\n",
    "    file_columns : dict\n",
    "        Dictionary containing the file paths of the csv-files in the sequences\n",
    "        path as keys and the column names of each csv-file as values.\"\"\"\n",
    "\n",
    "sequences, sequence_dict = helpers.crawl_sequences_data(path=datapackage_path_origin / \"data\" / \"sequences\")\n",
    "\n",
    "sequences.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c37a071-e311-477f-94d4-c0d750ec01dc",
   "metadata": {},
   "source": [
    "#### Specify and run tsam \n",
    "See also TSAM examples as Jupyter Notebooks: https://github.com/FZJ-IEK3-VSA/tsam/blob/master/examples/aggregation_example.ipynb\n",
    "And for all parameter: https://tsam.readthedocs.io/en/latest/timeseriesaggregationDoc.html#time-series-aggregation-class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "5e429c9c-b7a7-4c85-92c0-e058a0c3a0d6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tsam.timeseriesaggregation.TimeSeriesAggregation at 0x7d8191f8f250>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# aggregate time series by running tsam \n",
    "\n",
    "aggregation = tsam.TimeSeriesAggregation(sequences,\n",
    "    noTypicalPeriods=10,\n",
    "    hoursPerPeriod=24,\n",
    "    sortValues=False,\n",
    "    clusterMethod=\"k_means\",\n",
    "    rescaleClusterPeriods=False,\n",
    "    extremePeriodMethod=\"replace_cluster_center\",\n",
    "    representationMethod=\"durationRepresentation\")\n",
    "\n",
    "aggregation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f8f30cb-b492-48b4-987a-3c68b459552f",
   "metadata": {},
   "source": [
    "#### Prepare oemof-tabular datapackage for tsam"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18233032-ce59-41b8-b277-40ca3cab9753",
   "metadata": {},
   "source": [
    "##### Take tsa_aggregation object and derive tsa_parameters, tsa_sequences, tsa_timeindex.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    tsa_aggregation: tsam.TimeSeriesAggregation\n",
    "        contains all relevant parameters and values as a result of executing\n",
    "        the time series aggregation\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    tsa_sequences: pd.DataFrame\n",
    "        contains typical periods and data of all oemof-tabular sequences\n",
    "    tsa_parameters: pd.DataFrame\n",
    "        contains meta information to solph oemof model using tsam\n",
    "    tsa_timeindex: pd.Index\n",
    "        contains the timeindex of aggregated and is used as index for\n",
    "        seqeuences data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "ca05e448-81b7-4529-a79b-77a86be6274c",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "n_samples=5 should be >= n_clusters=10.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[72], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m aggregated_sequences, parameters, timeindex \u001b[38;5;241m=\u001b[39m \u001b[43mhelpers\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mprepare_oemof_parameters\u001b[49m\u001b[43m(\u001b[49m\u001b[43maggregation\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/git_repos/oemof_tsam_tutorial/oemof_tsam_helpers.py:143\u001b[0m, in \u001b[0;36mprepare_oemof_parameters\u001b[0;34m(tsa_aggregation)\u001b[0m\n\u001b[1;32m    120\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mprepare_oemof_parameters\u001b[39m(\n\u001b[1;32m    121\u001b[0m     tsa_aggregation: tsam\u001b[38;5;241m.\u001b[39mTimeSeriesAggregation,\n\u001b[1;32m    122\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28mtuple\u001b[39m[pd\u001b[38;5;241m.\u001b[39mDataFrame, pd\u001b[38;5;241m.\u001b[39mDataFrame, pd\u001b[38;5;241m.\u001b[39mIndex]:\n\u001b[1;32m    123\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m    124\u001b[0m \u001b[38;5;124;03m    Take tsa_aggregation object and derive tsa_parameters, tsa_sequences, tsa_timeindex.\u001b[39;00m\n\u001b[1;32m    125\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    141\u001b[0m \n\u001b[1;32m    142\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 143\u001b[0m     tsa_sequences \u001b[38;5;241m=\u001b[39m \u001b[43mtsa_aggregation\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreateTypicalPeriods\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    144\u001b[0m     tsa_sequences \u001b[38;5;241m=\u001b[39m pd\u001b[38;5;241m.\u001b[39mDataFrame(tsa_sequences)\n\u001b[1;32m    146\u001b[0m     tsa_parameters \u001b[38;5;241m=\u001b[39m {\n\u001b[1;32m    147\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mperiod\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;241m1\u001b[39m,\n\u001b[1;32m    148\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtimesteps_per_period\u001b[39m\u001b[38;5;124m\"\u001b[39m: tsa_aggregation\u001b[38;5;241m.\u001b[39mhoursPerPeriod,\n\u001b[1;32m    149\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124morder\u001b[39m\u001b[38;5;124m\"\u001b[39m: [tsa_aggregation\u001b[38;5;241m.\u001b[39mclusterOrder\u001b[38;5;241m.\u001b[39mtolist()],\n\u001b[1;32m    150\u001b[0m     }\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/tsam/timeseriesaggregation.py:1018\u001b[0m, in \u001b[0;36mTimeSeriesAggregation.createTypicalPeriods\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1011\u001b[0m cluster_duration \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mtime()\n\u001b[1;32m   1012\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msortValues:\n\u001b[1;32m   1013\u001b[0m     \u001b[38;5;66;03m# cluster the data\u001b[39;00m\n\u001b[1;32m   1014\u001b[0m     (\n\u001b[1;32m   1015\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mclusterCenters,\n\u001b[1;32m   1016\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mclusterCenterIndices,\n\u001b[1;32m   1017\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_clusterOrder,\n\u001b[0;32m-> 1018\u001b[0m     ) \u001b[38;5;241m=\u001b[39m \u001b[43maggregatePeriods\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1019\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcandidates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1020\u001b[0m \u001b[43m        \u001b[49m\u001b[43mn_clusters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnoTypicalPeriods\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1021\u001b[0m \u001b[43m        \u001b[49m\u001b[43mn_iter\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m100\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1022\u001b[0m \u001b[43m        \u001b[49m\u001b[43msolver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msolver\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1023\u001b[0m \u001b[43m        \u001b[49m\u001b[43mclusterMethod\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mclusterMethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1024\u001b[0m \u001b[43m        \u001b[49m\u001b[43mrepresentationMethod\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrepresentationMethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1025\u001b[0m \u001b[43m        \u001b[49m\u001b[43mrepresentationDict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrepresentationDict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1026\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdistributionPeriodWise\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdistributionPeriodWise\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1027\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtimeStepsPerPeriod\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimeStepsPerPeriod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1028\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1029\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1030\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mclusterCenters, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_clusterOrder \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_clusterSortedPeriods(\n\u001b[1;32m   1031\u001b[0m         candidates\n\u001b[1;32m   1032\u001b[0m     )\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/tsam/periodAggregation.py:68\u001b[0m, in \u001b[0;36maggregatePeriods\u001b[0;34m(candidates, n_clusters, n_iter, clusterMethod, solver, representationMethod, representationDict, distributionPeriodWise, timeStepsPerPeriod)\u001b[0m\n\u001b[1;32m     64\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mcluster\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m KMeans\n\u001b[1;32m     66\u001b[0m k_means \u001b[38;5;241m=\u001b[39m KMeans(n_clusters\u001b[38;5;241m=\u001b[39mn_clusters, max_iter\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1000\u001b[39m, n_init\u001b[38;5;241m=\u001b[39mn_iter, tol\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1e-4\u001b[39m)\n\u001b[0;32m---> 68\u001b[0m clusterOrder \u001b[38;5;241m=\u001b[39m \u001b[43mk_means\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfit_predict\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcandidates\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     69\u001b[0m \u001b[38;5;66;03m# get with own mean representation to avoid numerical trouble caused by sklearn\u001b[39;00m\n\u001b[1;32m     70\u001b[0m clusterCenters, clusterCenterIndices \u001b[38;5;241m=\u001b[39m representations(\n\u001b[1;32m     71\u001b[0m     candidates,\n\u001b[1;32m     72\u001b[0m     clusterOrder,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     77\u001b[0m     timeStepsPerPeriod\u001b[38;5;241m=\u001b[39mtimeStepsPerPeriod,\n\u001b[1;32m     78\u001b[0m )\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/sklearn/cluster/_kmeans.py:1070\u001b[0m, in \u001b[0;36m_BaseKMeans.fit_predict\u001b[0;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[1;32m   1047\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mfit_predict\u001b[39m(\u001b[38;5;28mself\u001b[39m, X, y\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m, sample_weight\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m):\n\u001b[1;32m   1048\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Compute cluster centers and predict cluster index for each sample.\u001b[39;00m\n\u001b[1;32m   1049\u001b[0m \n\u001b[1;32m   1050\u001b[0m \u001b[38;5;124;03m    Convenience method; equivalent to calling fit(X) followed by\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1068\u001b[0m \u001b[38;5;124;03m        Index of the cluster each sample belongs to.\u001b[39;00m\n\u001b[1;32m   1069\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m-> 1070\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msample_weight\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msample_weight\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241m.\u001b[39mlabels_\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/sklearn/base.py:1473\u001b[0m, in \u001b[0;36m_fit_context.<locals>.decorator.<locals>.wrapper\u001b[0;34m(estimator, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1466\u001b[0m     estimator\u001b[38;5;241m.\u001b[39m_validate_params()\n\u001b[1;32m   1468\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m config_context(\n\u001b[1;32m   1469\u001b[0m     skip_parameter_validation\u001b[38;5;241m=\u001b[39m(\n\u001b[1;32m   1470\u001b[0m         prefer_skip_nested_validation \u001b[38;5;129;01mor\u001b[39;00m global_skip_validation\n\u001b[1;32m   1471\u001b[0m     )\n\u001b[1;32m   1472\u001b[0m ):\n\u001b[0;32m-> 1473\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfit_method\u001b[49m\u001b[43m(\u001b[49m\u001b[43mestimator\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/sklearn/cluster/_kmeans.py:1473\u001b[0m, in \u001b[0;36mKMeans.fit\u001b[0;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[1;32m   1438\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Compute k-means clustering.\u001b[39;00m\n\u001b[1;32m   1439\u001b[0m \n\u001b[1;32m   1440\u001b[0m \u001b[38;5;124;03mParameters\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1462\u001b[0m \u001b[38;5;124;03m    Fitted estimator.\u001b[39;00m\n\u001b[1;32m   1463\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m   1464\u001b[0m X \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_validate_data(\n\u001b[1;32m   1465\u001b[0m     X,\n\u001b[1;32m   1466\u001b[0m     accept_sparse\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcsr\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1470\u001b[0m     accept_large_sparse\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m,\n\u001b[1;32m   1471\u001b[0m )\n\u001b[0;32m-> 1473\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_check_params_vs_input\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1475\u001b[0m random_state \u001b[38;5;241m=\u001b[39m check_random_state(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mrandom_state)\n\u001b[1;32m   1476\u001b[0m sample_weight \u001b[38;5;241m=\u001b[39m _check_sample_weight(sample_weight, X, dtype\u001b[38;5;241m=\u001b[39mX\u001b[38;5;241m.\u001b[39mdtype)\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/sklearn/cluster/_kmeans.py:1414\u001b[0m, in \u001b[0;36mKMeans._check_params_vs_input\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m   1413\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_check_params_vs_input\u001b[39m(\u001b[38;5;28mself\u001b[39m, X):\n\u001b[0;32m-> 1414\u001b[0m     \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_check_params_vs_input\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdefault_n_init\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m10\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1416\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_algorithm \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39malgorithm\n\u001b[1;32m   1417\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_algorithm \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124melkan\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mn_clusters \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m1\u001b[39m:\n",
      "File \u001b[0;32m~/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/sklearn/cluster/_kmeans.py:878\u001b[0m, in \u001b[0;36m_BaseKMeans._check_params_vs_input\u001b[0;34m(self, X, default_n_init)\u001b[0m\n\u001b[1;32m    875\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_check_params_vs_input\u001b[39m(\u001b[38;5;28mself\u001b[39m, X, default_n_init\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m):\n\u001b[1;32m    876\u001b[0m     \u001b[38;5;66;03m# n_clusters\u001b[39;00m\n\u001b[1;32m    877\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m X\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m0\u001b[39m] \u001b[38;5;241m<\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mn_clusters:\n\u001b[0;32m--> 878\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m    879\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mn_samples=\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mX\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m should be >= n_clusters=\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mn_clusters\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    880\u001b[0m         )\n\u001b[1;32m    882\u001b[0m     \u001b[38;5;66;03m# tol\u001b[39;00m\n\u001b[1;32m    883\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_tol \u001b[38;5;241m=\u001b[39m _tolerance(X, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtol)\n",
      "\u001b[0;31mValueError\u001b[0m: n_samples=5 should be >= n_clusters=10."
     ]
    }
   ],
   "source": [
    "aggregated_sequences, parameters, timeindex = helpers.prepare_oemof_parameters(aggregation)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "244c8ca2-cb91-44c3-8405-19dd7e2d1ee4",
   "metadata": {},
   "source": [
    "##### Convert and save aggregated time series dataframe into oemof-tabular sequence files\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    tsa_sequences: pd.DataFrame\n",
    "        contains typical periods and data of all oemof-tabular sequences\n",
    "\n",
    "    tsa_timeindex: pd.Index\n",
    "        contains the timeindex of aggregated and is used as index for\n",
    "        seqeuences data.\n",
    "\n",
    "    file_columns : dict\n",
    "        Dictionary containing the file paths of the csv-files in the sequences\n",
    "        path as keys and the column names of each csv-file as values.\n",
    "\n",
    "    path : Path\n",
    "        The Path object pointing to the oemof-tabular sequences directory\n",
    "        (data/sequences) in which the tsa_profiles will be stored.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "1665714f-3455-4a2d-8023-b6fd82d4333e",
   "metadata": {},
   "outputs": [],
   "source": [
    "helpers.convert_tsa_sequences_to_oemof_sequences(aggregated_sequences, timeindex, sequence_dict, path=datapackage_path_tsam / \"data\" / \"sequences\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1834b3cf-f00b-4f48-9db8-a2d74bfb0e85",
   "metadata": {},
   "source": [
    "##### Store tsa_parameters to path.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    tsa_parameters : pd.DataFrame\n",
    "        The path to the origin directory to copy from. Defaults to elements_original_path.\n",
    "    path : Path\n",
    "        The path to the oemof-tabular datapackage data/tsam.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "f6c378d4-4647-4edc-a0a8-f2556e2c9cf7",
   "metadata": {},
   "outputs": [],
   "source": [
    "helpers.store_tsa_parameter(parameters, path=datapackage_path_tsam / \"data\" / \"tsam\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62fc8df3-de9d-4042-a540-320b5e4e6dfc",
   "metadata": {},
   "source": [
    "##### Create and store periods into oemof-tabular datapackage.\n",
    "\n",
    "This necessary for multi-period optimization in oemof, if no_of_periods=0 function passes None.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    tsa_timeindex: pd.Index\n",
    "        Contains the timeindex of aggregated and is used as index for\n",
    "        seqeuences data.\n",
    "\n",
    "    no_of_periods : int\n",
    "        Number of periods used in oemof NOT in time series aggregation.\n",
    "\n",
    "    timeincrement : int\n",
    "        Timeincrement for each period and timestep to allow for\n",
    "        segmentation.\n",
    "\n",
    "    path : Path, Default: data/periods\n",
    "        The Path object pointing to the tsam oemof-tabular datapackage directory\n",
    "        in which the periods will be stored.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    periods: pd.DataFrame\n",
    "        Dataframe that maps timeindex, periods and timeincrement for\n",
    "        each period."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "2b49a42b-8e68-4595-86ce-aaaff3cbf0ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "helpers.create_oemof_periods_csv(timeindex, path=datapackage_path_tsam / \"data\"/ \"periods\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22b36db2-ed5b-4758-a00c-54b0e475d8a6",
   "metadata": {},
   "source": [
    "##### Copy data from the origin directory to the goal directory if the goal directory does not exist.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    origin_path : Union[str, Path]\n",
    "        The path to the origin directory to copy from. Defaults to elements_original_path.\n",
    "    goal_path : Union[str, Path]\n",
    "        The path to the goal directory to copy to. Defaults to elements_path.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "70ad9ca3-847c-45ca-ada6-f624b5ebbc92",
   "metadata": {},
   "outputs": [],
   "source": [
    "helpers.copy_elements_data(origin_path=datapackage_path_origin/ \"data\"/ \"elements\", goal_path=datapackage_path_tsam / \"data\" / \"elements\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5557fb17-84ed-4a1b-a8d5-0ebb5fb64dbb",
   "metadata": {},
   "source": [
    "##### Copy and append datapackage.json\n",
    "\n",
    "from the origin directory to the goal directory if the goal directory does not exist. \n",
    "\n",
    "    \"\"\"\n",
    "    resource= {\n",
    "            \"path\": \"data/tsam/tsa_parameters.csv\",\n",
    "            \"profile\": \"tabular-data-resource\",\n",
    "            \"name\": \"tsa_parameters\",\n",
    "            \"format\": \"csv\",\n",
    "            \"mediatype\": \"text/csv\",\n",
    "            \"encoding\": \"utf-8\",\n",
    "            \"schema\": {\n",
    "                \"fields\": [\n",
    "                    {\n",
    "                        \"name\": \"period\",\n",
    "                        \"type\": \"integer\",\n",
    "                        \"format\": \"default\"\n",
    "                    },\n",
    "                    {\n",
    "                        \"name\": \"timesteps_per_period\",\n",
    "                        \"type\": \"integer\",\n",
    "                        \"format\": \"default\"\n",
    "                    },\n",
    "                    {\n",
    "                        \"name\": \"order\",\n",
    "                        \"type\": \"array\",\n",
    "                        \"format\": \"default\"\n",
    "                    }\n",
    "                ],\n",
    "                \"missingValues\": [\n",
    "                    \"\"\n",
    "                ]\n",
    "            }\n",
    "        }\n",
    "    \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "1b82b5a6-5a57-42e0-98bd-f126cab3cbaf",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sarah/git_repos/oemof_tsam_tutorial/oemof_tsam_helpers.py:432: UserWarning: Datapackage.json already exits in goal_path. Please make sure the tsam resource is in datapackage.json.\n",
      "  warnings.warn(\"Datapackage.json already exits in goal_path. \"\n"
     ]
    }
   ],
   "source": [
    "helpers.copy_and_append_datapackage_json(origin_path= datapackage_path_origin, goal_path=datapackage_path_tsam)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87b91fe2-f385-4dce-bda5-23fac29b2693",
   "metadata": {},
   "source": [
    "#### Create oemof-Model from datapackage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "543e2f3c-3ba2-4345-85ec-bcb21b7e00dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "scenario_name=datapackage_name_tsam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "15bdddd9-4c8b-4104-b105-91baa4451078",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/sarah/git_repos/oemof_tsam_tutorial/dispatch_tsam/results\n"
     ]
    }
   ],
   "source": [
    "datapackage_path=Path(Path.cwd(),scenario_name)\n",
    "results_path=Path(datapackage_path,'results')\n",
    "\n",
    "if not results_path.exists():\n",
    "    Path.mkdir(results_path)\n",
    "\n",
    "print(results_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "e2d9f13b-c351-45a3-bf16-7150da378965",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/tabular/datapackage/reading.py:152: UserWarning: Version of datapackage '0.0.6dev' is not supported. These versions are supported: [None, '0.0.1', '0.0.2', '0.0.3', '0.0.4', '0.0.5dev']\n",
      "  warnings.warn(\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/_energy_system.py:124: FutureWarning: The default behaviour will change in future versions.\n",
      "At the moment the last interval of an equidistant time index is added implicitly by default. Set 'infer_last_interval' explicitly 'True' or 'False' to avoid this warning. In future versions 'False' will be the defaultbehaviour\n",
      "  warnings.warn(msg, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# create energy system object\n",
    "es = EnergySystem.from_datapackage(\n",
    "    os.path.join(datapackage_path, \"datapackage.json\"),\n",
    "    attributemap={},\n",
    "    typemap=TYPEMAP,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "c4202ef6-ca9f-4baa-b66f-a02c2b963936",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:508: FutureWarning: DataFrame.interpolate with method=pad is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  flow_dict[flow] = ts.interpolate(method=\"pad\").reset_index(\"timestep\")\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:636: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  interpolated_soc = soc_ts.interpolate()\n",
      "/home/sarah/miniconda3/envs/oemof_dev_2024/lib/python3.10/site-packages/oemof/solph/processing.py:636: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  interpolated_soc = soc_ts.interpolate()\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "5.034213304519653"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create model from energy system (this is just oemof.solph)\n",
    "m = Model(es)\n",
    "\n",
    "# add constraints from datapackage to the model\n",
    "m.add_constraints_from_datapackage(\n",
    "    os.path.join(datapackage_path, \"datapackage.json\"),\n",
    "    constraint_type_map=CONSTRAINT_TYPE_MAP,\n",
    ")\n",
    "\n",
    "# if you want dual variables / shadow prices uncomment line below\n",
    "# m.receive_duals()\n",
    "\n",
    "startTime = time.time()\n",
    "# select solver 'gurobi', 'cplex', 'glpk' etc\n",
    "m.solve('cbc')\n",
    "executionTime = time.time() - startTime\n",
    "\n",
    "es.params = processing.parameter_as_dict(es)\n",
    "es.results = m.results()\n",
    "\n",
    "# now we use the write results method to write the results in oemof-tabular\n",
    "# format\n",
    "postprocessed_results = calculations.run_postprocessing(es)\n",
    "postprocessed_results.to_csv(os.path.join(results_path, \"results.csv\"))\n",
    "executionTime"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34289caa-ec5b-4cf7-a83d-41bc4526f482",
   "metadata": {},
   "source": [
    "#### Compare results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "07d07694-0dc9-4bc9-b7d5-dc86567ff420",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>var_name</th>\n",
       "      <th>var_value</th>\n",
       "      <th>var_value_tsam</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>coal</td>\n",
       "      <td>flow_out_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>coal</td>\n",
       "      <td>summed_variable_costs_out_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>gas</td>\n",
       "      <td>flow_out_bus1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>gas</td>\n",
       "      <td>summed_variable_costs_out_bus1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>lignite</td>\n",
       "      <td>flow_out_bus0</td>\n",
       "      <td>5981.578947</td>\n",
       "      <td>5981.578933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>lignite</td>\n",
       "      <td>summed_variable_costs_out_bus0</td>\n",
       "      <td>119631.578946</td>\n",
       "      <td>119631.578669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>el-storage1</td>\n",
       "      <td>flow_in_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>el-storage2</td>\n",
       "      <td>flow_in_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>el-storage1</td>\n",
       "      <td>flow_out_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>el-storage2</td>\n",
       "      <td>flow_out_bus0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>el-storage1</td>\n",
       "      <td>storage_losses</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>el-storage2</td>\n",
       "      <td>storage_losses</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>pv</td>\n",
       "      <td>flow_out_bus1</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>wind</td>\n",
       "      <td>flow_out_bus0</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>demand0</td>\n",
       "      <td>flow_in_bus0</td>\n",
       "      <td>5000.000000</td>\n",
       "      <td>5000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>conn1</td>\n",
       "      <td>flow_in_bus0_from_bus</td>\n",
       "      <td>1031.578948</td>\n",
       "      <td>1031.578946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>demand1</td>\n",
       "      <td>flow_in_bus1</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>conn1</td>\n",
       "      <td>flow_in_bus1_to_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>conn2</td>\n",
       "      <td>flow_in_bus_from_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>conn2</td>\n",
       "      <td>flow_in_bus_to_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>conn1</td>\n",
       "      <td>flow_out_bus0_from_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>conn1</td>\n",
       "      <td>flow_out_bus1_to_bus</td>\n",
       "      <td>979.999999</td>\n",
       "      <td>979.999997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>conn2</td>\n",
       "      <td>flow_out_bus_from_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>conn2</td>\n",
       "      <td>flow_out_bus_to_bus</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>system</td>\n",
       "      <td>total_system_cost</td>\n",
       "      <td>119631.578946</td>\n",
       "      <td>119631.578669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>conn1</td>\n",
       "      <td>transmission_losses</td>\n",
       "      <td>51.578949</td>\n",
       "      <td>51.578949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>conn2</td>\n",
       "      <td>transmission_losses</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           name                        var_name      var_value  var_value_tsam\n",
       "0          coal                   flow_out_bus0       0.000000        0.000000\n",
       "1          coal  summed_variable_costs_out_bus0       0.000000        0.000000\n",
       "2           gas                   flow_out_bus1       0.000000        0.000000\n",
       "3           gas  summed_variable_costs_out_bus1       0.000000        0.000000\n",
       "4       lignite                   flow_out_bus0    5981.578947     5981.578933\n",
       "5       lignite  summed_variable_costs_out_bus0  119631.578946   119631.578669\n",
       "6   el-storage1                    flow_in_bus0       0.000000        0.000000\n",
       "7   el-storage2                    flow_in_bus0       0.000000        0.000000\n",
       "8   el-storage1                   flow_out_bus0       0.000000        0.000000\n",
       "9   el-storage2                   flow_out_bus0       0.000000        0.000000\n",
       "10  el-storage1                  storage_losses       0.000000        0.000000\n",
       "11  el-storage2                  storage_losses       0.000000        0.000000\n",
       "12           pv                   flow_out_bus1      20.000000       20.000000\n",
       "13         wind                   flow_out_bus0      50.000000       50.000000\n",
       "14      demand0                    flow_in_bus0    5000.000000     5000.000000\n",
       "15        conn1           flow_in_bus0_from_bus    1031.578948     1031.578946\n",
       "16      demand1                    flow_in_bus1    1000.000000     1000.000000\n",
       "17        conn1             flow_in_bus1_to_bus       0.000000        0.000000\n",
       "18        conn2            flow_in_bus_from_bus       0.000000        0.000000\n",
       "19        conn2              flow_in_bus_to_bus       0.000000        0.000000\n",
       "20        conn1          flow_out_bus0_from_bus       0.000000        0.000000\n",
       "21        conn1            flow_out_bus1_to_bus     979.999999      979.999997\n",
       "22        conn2           flow_out_bus_from_bus       0.000000        0.000000\n",
       "23        conn2             flow_out_bus_to_bus       0.000000        0.000000\n",
       "24       system               total_system_cost  119631.578946   119631.578669\n",
       "25        conn1             transmission_losses      51.578949       51.578949\n",
       "26        conn2             transmission_losses       0.000000        0.000000"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_origin=pd.read_csv(datapackage_path_origin / \"results\"/ \"results.csv\")\n",
    "results_tsam= pd.read_csv(datapackage_path_tsam / \"results\" / \"results.csv\")\n",
    "\n",
    "results_origin['var_value_tsam']=results_tsam['var_value']\n",
    "results_origin.drop(columns=['region','type','carrier','tech'], inplace=True)\n",
    "results_origin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92fa1bf8-8293-455e-9758-a7ff4d55f3bc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}