MJ2383_Lab_3.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b9EKJZFJXxja"
      },
      "source": [
        "**Note: Run the following cell if you are working in Google Colab!**\n",
        "\n",
        "This will install:\n",
        "* `glpk` (GNU Linear Programming Kit), which is a package intended for solving large-scale linear programming (LP), mixed integer programming (MIP), and other related problems.\n",
        "* `otoole`, or **O**SeMOSYS **tool**s for **e**nergy work, which provides commonly used pre- and post-processing steps when working with OSeMOSYS models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SVb1Y08T86ZA"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/KTH-dESA/MJ2383.git\n",
        "!mv MJ2383/* .\n",
        "!rm -r MJ2383\n",
        "!pip install -q condacolab\n",
        "import condacolab\n",
        "condacolab.install()\n",
        "!conda install -c conda-forge glpk\n",
        "!pip install otoole"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W--MkMnF9v1t"
      },
      "source": [
        "Don't forget to run the imports!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RJdlrBbs8vmx",
        "tags": []
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "from plotly import express as px\n",
        "from tqdm import trange\n",
        "import os\n",
        "from google.colab import files"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F0yKj2oUJr0C"
      },
      "source": [
        "# MJ2383 Computer Lab 3\n",
        "\n",
        "This lab aims to provide an inside view on what a supply curve is and how it can be generated while using OSeMOSYS. Reminder, a supply curve is a graphical representation of the law of supply. It slopes upward because the quantity supplied increases as price increases, with other things constant.\n",
        "\n",
        "In this lab, we'll be using OSeMOSYS, but we'll be running it in the background using this Jupyter Notebook to control the input data, run the model and extract and visualise the results.\n",
        "\n",
        "## Editing data\n",
        "We will be using three OSeMOSYS models of increasing complexity during this lab, `gas`, `gas_more` and `gas_solar_wind`. The inputs to these models are stored in three excel workbooks (available on **model** folder in the files tab in the left panel). During the lab you will have to edit certain parameters of the models, located in specific sheets, and upload to your session the edited file. This is done either by openning the left pannel on the `Files` tab (the **folder** icon) and dragging and droping your excel file in the `model` folder, or by running the respective cells to upload the models.You will find those along the notebook calling the next function:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def upload_model(model):\n",
        "    os.chdir('model')\n",
        "    if os.path.exists(f'{model}.xlsx'):\n",
        "        os.remove(f'{model}.xlsx')\n",
        "    files.upload()\n",
        "    os.chdir('..')"
      ],
      "metadata": {
        "id": "B2d6O44dFUNF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7t2ADRow8vmz"
      },
      "source": [
        "## Units\n",
        "The units of the different input data are:\n",
        "\n",
        "Parameter | Unit\n",
        ":-- | --:\n",
        "Demand | PJ\n",
        "Capacity | GW\n",
        "Activity | PJ\n",
        "Capital Cost | M\\$/GW\n",
        "Fixed Cost | M\\$/GW\n",
        "Variable Cost, Fossil Fuels Extraction/Import | M\\$/PJ\n",
        "Variable Cost, Renewables | M\\$/GWh\n",
        "Operational Life | Yr\n",
        "Emission Activity Ratio | MtCO2/PJ\n",
        "Annual Emission Limit | MtCO2\n",
        "Emission Penalty | \\$/tCO2\n",
        "\n",
        "## Parameters\n",
        "\n",
        "**If you are not yet familiar with OSeMOSYS parameters, open the [manual](https://osemosys.readthedocs.io/en/latest/manual/Structure%20of%20OSeMOSYS.html#parameters) in a browser window for reference throughout the lab.**\n",
        "\n",
        "## Running a model\n",
        "\n",
        "Running a model is a two step process. We will explain the steps here, and use them later in the notebook. Firstly, you need to convert your excel data into an OSeMOSYS datafile with the next line following:\n",
        "\n",
        "    !otoole convert excel datafile gas.xlsx temp.txt\n",
        "    \n",
        "Note you need the `!` prepended to the command (this tells the notebook to run this command using the shell rather than Python). Here we create datafile called `temp.txt` from the model data stored in `gas.xlsx`.\n",
        "\n",
        "After generating the OSeMOSYS datafile (in this case, called `temp.txt`) we then solve the model using `glpsol`, which is the open-source solver provided by the GNU Math Programming kit.\n",
        "\n",
        "    !glpsol -d temp.txt -m osemosys.txt > osemosys.log\n",
        "    \n",
        "The results are then available in the `results` folder. You should also check the `osemosys.log` file to ensure that the model ran correctly.\n",
        "\n",
        "In the stages below, we load data from the model inputs and results using a Python library called `pandas` and plot them in interactive charts.\n",
        "\n",
        "## Contents\n",
        "\n",
        "- [Stage 1](#s1) - In this section, we explore a very simple model with a two-stage supply curve for natural gas to develop our economic interpretation of OSeMOSYS\n",
        "- [Stage 2](#s2) - We add complexity, by increasing the number of steps in our supply curve by adding resources. We explore what difference this makes to the electricity price under different conditions.\n",
        "- [Stage 3](#s3) - We add an emissions penalty, imposing a tax upon CO2\n",
        "- [Stage 4](#s4) - We add renewable technologies to the electricity sector, whose marginal cost of generation is 0. However, this creates a demand for backup capacity. How does this affect the marginal price of electricity?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pTOBJuYG8vm1"
      },
      "source": [
        "<a name=\"s1\"></a>\n",
        "\n",
        "## Stage 1: A super simple supply curve\n",
        "First, we will download the data of the `gas` model, by running the next cell:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "files.download('model/gas.xlsx')"
      ],
      "metadata": {
        "id": "HJhGRN5w7GqV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4pgOQ0MHvkgg"
      },
      "source": [
        "We start with the simplest supply cost curve you can imagine. In this simple OSeMOSYS model, there are two natural gas commodities (fuels) - imports of natural gas `GasImport`, and extraction `GasExtraction`. Both of these generate CO2, and feed a natural gas combined-cycle electricity generation plant `NGCC`. This produces secondary electricity `SEC_EL` which enters a transmission and distribution technology `TD` which produces final electricity `FEL`.\n",
        "\n",
        "![](https://github.com/KTH-dESA/MJ2383/blob/main/img/simple.svg?raw=1)\n",
        "\n",
        "(Note, this image is produced using the command `!otoole viz res excel gas.xlsx simple.svg config.yaml`)\n",
        "\n",
        "---\n",
        "\n",
        "First, let's have a think about the supply curve in this system.\n",
        "\n",
        "Shown on the left of the image, there are two natural gas resources. `GasExtraction` has a maximum production capacity `TotalAnnualMaxCapacity` (see the sheet `TotalAnnualMaxCapacity` in the excel file) of 6 PJ/year at a variable cost `VariableCost` (see the excel sheet `VariableCost`) of €8/PJ. However, `GasImport` has no upper bound, but a higher cost of €12/PJ.\n",
        "\n",
        "Plotted, this looks rather uninspiring, but at least gives an impression of a supply curve for natural gas.\n",
        "\n",
        "__Run the next cell to view the graph__"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "53Pifebm8vm1"
      },
      "outputs": [],
      "source": [
        "gas_extraction = [[\"GasExtraction\", x, 8] for x in range(7)]\n",
        "gas_import = [[\"GasImport\", x, 12] for x in range(6, 21)]\n",
        "gas_supply_curve = pd.DataFrame(gas_extraction + gas_import,\n",
        "    columns=['name', 'supply', 'marginal_cost'])\n",
        "fig = px.line(gas_supply_curve, x='supply', y='marginal_cost', range_x=[0, 20], range_y=[0, 20], line_shape='vh')\n",
        "fig.update_traces(mode=\"markers+lines\")\n",
        "fig.update_xaxes(showspikes=True)\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lIkwg0qK8vm1"
      },
      "source": [
        "In OSeMOSYS, supply of energy must equal demand for energy. As the quantity of energy demanded increases, energy supply increases accordingly. Given that our demand curve is a straight line (imagine a vertical line moving along the x-axis) the equilibrium point (where the lines cross) denotes the marginal cost of production and in this case is equal to the price.\n",
        "\n",
        "Within most energy systems, there are multiple markets for different, related, energy commodities. Later, we will use our simple OSeMOSYS model to begin to understand the interactions between these different energy markets."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AiBfDD-38vm1"
      },
      "source": [
        "### Exercise 1: Manually computing the marginal cost of electricity\n",
        "\n",
        "Think about how our simple supply curve for natural gas interacts with the demand for electricity.\n",
        "\n",
        "In our simple example, we demand a specific quantity of electricity. The electricity is produced by the `NGCC` technology which requires almost 2 units of gas to produce 1 unit of electricity.\n",
        "\n",
        "**Now answer the quiz question 1 on Canvas.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5roaiL_18vm2"
      },
      "source": [
        "### Marginal Prices\n",
        "\n",
        "In OSeMOSYS, there is a key equation called a \"balancing constraint\". This ensures that energy is conserved in each timeslice.\n",
        "\n",
        "```ampl\n",
        "s.t. EBa11_EnergyBalanceEachTS5{r in REGION, l in TIMESLICE, f in FUEL, y in YEAR}:\n",
        "\tProduction[r,l,f,y] >= Demand[r,l,f,y] + Use[r,l,f,y]\n",
        "\t+ sum{rr in REGION} Trade[r,rr,l,f,y] * TradeRoute[r,rr,f,y];\n",
        "```\n",
        "\n",
        "We can extract some useful information from this equation when we solve the optimisation problem to minimise costs.  These data are stored in the file `results/ProductionDual.csv`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NohfkcLH8vm2"
      },
      "source": [
        "### Exercise 2: Understanding production\n",
        "\n",
        "We'll now run OSeMOSYS from this Jupyter Notebook to compute the equilibrium price. But first we will add some demand data to the `SpecifiedAnnualDemand` sheet. Open the `gas.xlsx` model, and in the `SpecifiedAnnualDemand` sheet change the demand to 7. **All sheets are in alphabetic order**.\n",
        "\n",
        "Now run the next cell to upload the edited model and place it in the right place:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_model('gas')"
      ],
      "metadata": {
        "id": "fMqmodwI8wSq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X5mRRAl28vm2"
      },
      "source": [
        "Now we run OSeMOSYS and plot the total production by fuel over the year.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w1nrdSKt8vm2"
      },
      "outputs": [],
      "source": [
        "!otoole convert excel datafile model/gas.xlsx temp.txt config.yaml\n",
        "# Solve the model, writing the results into ./results\n",
        "os.makedirs('results', exist_ok=True)\n",
        "!glpsol -d temp.txt -m osemosys.txt > osemosys.log\n",
        "production = pd.read_csv('results/ProductionByTechnologyAnnual.csv').groupby(by=['TECHNOLOGY', 'FUEL']).sum()\n",
        "px.bar(production.reset_index(), x='FUEL', y='VALUE', color='TECHNOLOGY', labels={'VALUE': 'Energy (PJ)'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3PwWCRK8vm3"
      },
      "source": [
        "In reality, however, production differs across the year - in OSeMOSYS we use `TIMESLICES` to represent the fractions of the year in which different levels of demand exist. This approximates the average demand profile for electricity and other fuels across days and seasons. In our example model, we have defined 6 time-slices:\n",
        "\n",
        "TIMESLICE | Description\n",
        ":--|:--\n",
        "ID | Intermediate day\n",
        "IN | Intermediate night\n",
        "SD | Summer day\n",
        "SN | Summer night\n",
        "WD | Winter day\n",
        "WN | Winter night"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tPWaClF_8vm3"
      },
      "outputs": [],
      "source": [
        "production = pd.read_csv('results/ProductionByTechnology.csv').groupby(by=['TIMESLICE', 'FUEL']).sum()\n",
        "fig = px.bar(production.reset_index(), x='TIMESLICE', y='VALUE', color='FUEL',\n",
        "             category_orders={\"FUEL\": [\"GAS\", \"SEC_EL\", \"FEL\"]}, labels={'VALUE': 'Energy (PJ)'})\n",
        "fig.update_layout(barmode='group')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "okuv7Lyl8vm3"
      },
      "source": [
        "**Now answer quiz question 2 on Canvas**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SaR-eqWG8vm3"
      },
      "source": [
        "### Exercise 3: Using OSeMOSYS to compute the marginal price\n",
        "\n",
        "Below, we use a Python library called `pandas` to read the `ProductionDual` comma-separated values file. We extract the data point for `GAS` during the winter day (`WD`) timeslice."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3EMUhHB58vm3"
      },
      "outputs": [],
      "source": [
        "cost = pd.read_csv('results/ProductionDual.csv').set_index(['TIMESLICE', 'FUEL'])\n",
        "cost['DUAL'] = cost['DUAL'] / (1.05**-0.5)\n",
        "cost.loc[('WD', 'GAS'),'DUAL']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ec5vWL4H8vm4"
      },
      "source": [
        "By running the cell, we see that the marginal cost of producing gas, is equal to the cost of importing natural gas: about €12/PJ.  Interesting.  That must mean that the model is importing gas, rather than extracting it (which is cheaper).\n",
        "\n",
        "NB: The division of `(1.05**-0.5)` compounds the marginal value back to the base year. In OSeMOSYS, fixed and variable operating costs are discounted to the middle of the year.\n",
        "\n",
        "**Now answer quiz question 3 on Canvas. Hint: Consider tweaking and re-running selected parts of above code.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D4U2FTJm8vm4"
      },
      "source": [
        "Plotted onto the supply curve we plotted earlier, we see that the results from the OSeMOSYS model make sense. The marginal cost of gas production from the OSeMOSYS model sits on the supply cost curve of the two gas resources.\n",
        "\n",
        "However, we only have one point for electricity. In the next stages, we will use the model to compute the marginal cost of electricity production."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BMHm2YGt8vm4"
      },
      "outputs": [],
      "source": [
        "fig = px.line(gas_supply_curve, x='supply', y='marginal_cost',\n",
        "              range_x=[0, 20], range_y=[0, 30], line_shape='vh')\n",
        "\n",
        "gas_demand = production.groupby(by='FUEL').sum().loc['GAS', 'VALUE']\n",
        "elec_demand = production.groupby(by='FUEL').sum().loc['FEL', 'VALUE']\n",
        "\n",
        "fig.add_shape( # adds a vertical \"demand\" line\n",
        "    type=\"line\", line_color=\"salmon\", line_width=3, opacity=1, line_dash=\"dot\",\n",
        "    x0=gas_demand, x1=gas_demand, xref=\"x\", y0=0, y1=cost.loc[('WD', 'GAS'),'DUAL'])\n",
        "\n",
        "fig.add_annotation( # adds a text callout with arrow\n",
        "    text=\"Marginal cost of gas\", x=gas_demand, y=cost.loc[('WD', 'GAS'),'DUAL'],\n",
        "    arrowhead=1, showarrow=True\n",
        ")\n",
        "\n",
        "fig.add_annotation( # adds a text callout with arrow\n",
        "    text=\"Marginal cost of electricity\", x=elec_demand,\n",
        "    y=cost.loc[('WD', 'FEL'),'DUAL'], arrowhead=1, showarrow=True\n",
        ")\n",
        "\n",
        "fig.add_shape( # adds a vertical \"demand\" line\n",
        "    type=\"line\", line_color=\"salmon\", line_width=3, opacity=1, line_dash=\"dot\",\n",
        "    x0=elec_demand, x1=elec_demand, xref=\"x\", y0=0, y1=cost.loc[('WD', 'FEL'),'DUAL'])\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EiCAlcKt8vm4"
      },
      "source": [
        "### Notes\n",
        "\n",
        "- The x-axis shows the supply quantity of energy\n",
        "- The marginal cost of electricity is plotted for reference against the quantity of electricity supplied, and marginal cost of gas against the quantity of gas supplied"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U3c4oVLL8vm5"
      },
      "source": [
        "### Running the model to extract the marginal cost of production of electricity\n",
        "\n",
        "We can automate the running of the OSeMOSYS model over multiple levels of demand for electricity. The following code loops over demands ranging from 1 to *n* in increments of *x*.  We write the demand into the sheet  `SpecifiedAnnualDemand`, create the modelfile and then solve the model. We then extract the results from the `results/ProductionDual.csv` file and store them in a list.  Finally, the function returns the list of results for further processing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dKyetzGr8vm5",
        "tags": []
      },
      "outputs": [],
      "source": [
        "def write_demand(model, value):\n",
        "    \"\"\"Writes a demand value into the passed model\"\"\"\n",
        "    model_data = pd.read_excel(model, sheet_name=None)\n",
        "    model_data['SpecifiedAnnualDemand'][2014] = value\n",
        "    with pd.ExcelWriter(model) as writer:\n",
        "        for name, df in model_data.items():\n",
        "            df.to_excel(writer, sheet_name=name, index=False)\n",
        "\n",
        "def extract_result(param: str, production):\n",
        "    \"\"\"Read the marginal cost values from the results\n",
        "    \"\"\"\n",
        "    marginal = pd.read_csv('results/ProductionDual.csv'\n",
        "                           ).set_index(['TIMESLICE', 'FUEL'])\n",
        "    marg_ann = pd.read_csv('results/ProductionDualAnnual.csv'\n",
        "                           ).set_index(['FUEL'])\n",
        "    # production = pd.read_csv('results/ProductionByTechnology.csv').groupby(by=['TIMESLICE', 'FUEL']).sum()\n",
        "    try:\n",
        "        demand = production.groupby(by='FUEL').sum().loc[param, 'VALUE']\n",
        "    except KeyError:\n",
        "        demand = 0\n",
        "    try:\n",
        "        value = marginal.loc[('WD', param), 'DUAL'] / (1.05**-0.5)\n",
        "    except KeyError:\n",
        "        try:\n",
        "            value = marg_ann.loc[param, 'DUAL'] / (1.05**-0.5)\n",
        "        except KeyError:\n",
        "            value = 0\n",
        "    observation = {'value': value, 'param': param, 'quantity': demand}\n",
        "    return observation\n",
        "\n",
        "def run_model(model: str, start: int, stop: int, step=int):\n",
        "    \"\"\"Run the model for a range of demand values\n",
        "\n",
        "    Returns\n",
        "    -------\n",
        "    List\n",
        "        A list of dual values for GAS, FEL and COA\n",
        "    \"\"\"\n",
        "    results = []\n",
        "    model = f'model/{model}.xlsx'\n",
        "    for dem in trange(start, stop, step):\n",
        "        observation = {}\n",
        "        # data = {\"REGION\": 'SIMPLICITY', \"FUEL\": 'FEL', \"YEAR\": 2014, \"VALUE\": dem}\n",
        "        # add_data(model=model, dataset='SpecifiedAnnualDemand', data=data)\n",
        "        write_demand(model, dem)\n",
        "        !otoole convert excel datafile $model temp.txt config.yaml\n",
        "        # Solve the model, writing the results into ./results\n",
        "        os.makedirs('results', exist_ok=True)\n",
        "        !glpsol -d temp.txt -m osemosys.txt > osemosys.log\n",
        "        production = pd.read_csv('results/ProductionByTechnologyAnnual.csv').groupby(by=['TECHNOLOGY', 'FUEL']).sum()\n",
        "        # fuels = pd.read_csv(os.path.join('model', model, 'data', 'FUEL.csv'))\n",
        "        fuels = pd.read_excel(model, sheet_name='FUEL')\n",
        "        for fuel in fuels['VALUE'].values:\n",
        "            if fuel != 'SEC_EL':\n",
        "                results.append(extract_result(fuel, production))\n",
        "\n",
        "    return results"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Run the model:"
      ],
      "metadata": {
        "id": "eqizkMZT9iKp"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vOXBWzZX8vm5",
        "tags": []
      },
      "outputs": [],
      "source": [
        "gas_results = run_model(\"gas\", 1, 20, 1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S8Ziawd08vm5"
      },
      "outputs": [],
      "source": [
        "gas_data = pd.DataFrame(gas_results)\n",
        "px.line(gas_data, x='quantity', y='value', facet_col='param', range_x=[0, 20], range_y=[0, 40],\n",
        "        line_shape='vh', labels={'value': 'Cost'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tABa7eMn8vm6"
      },
      "source": [
        "In the plot above, we now see the supply cost curves derived from the OSeMOSYS model. Given the simple supply cost curve for gas, with two stages and a breakpoint between 4.19 and 6.29 (we know that the cost of gas production changes at 6), you can see that the marginal cost of producing 2 units of electricity is 16.77 and the marginal cost of producing 3 or more units is 25.16."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vs3GuIYb8vm6"
      },
      "source": [
        "## Summary of Stage 1\n",
        "\n",
        "In this first part of the lab we learned the following:\n",
        "\n",
        "- A simple supply curve can be constructed in OSeMOSYS using one technology per resource step. Parameters `TotalAnnualMaxCapacity` and `VariableCost` may be used to define each step in the supply cost curve for a resource.\n",
        "- The OSeMOSYS code can be extended to write out marginal costs for all commodities in the model. This is particularly useful to show the marginal cost of derived commodities, such as electricity, which are produced from a variety of different energy chains.\n",
        "- By running our model at different levels of demand, we can use the model to compute the derived supply cost curve for each commodity.\n",
        "- We can use OSeMOSYS to explore the relationship between the structure of the energy system, supply cost curves for primary resources, and supply cost curves for secondary and tertiary energy commodities."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='s2'></a>\n",
        "\n",
        "# Stage 2: Adding Resources\n",
        "First, download the model data `gas_more.xlsx` for this stage running the next cell:"
      ],
      "metadata": {
        "id": "NzqRieFJAGIU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "files.download('model/gas_more.xlsx')"
      ],
      "metadata": {
        "id": "k83MEjVrAHOp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KYRABfwX8vm6"
      },
      "source": [
        "In this part of the lab, we add a number of resources to our model and explore how this affects the supply curve for electricity. We now move towards a slightly more realistic energy system than the simplified model we used to introduce the concepts in Stage 1.\n",
        "\n",
        "![](https://github.com/KTH-dESA/MJ2383/blob/main/img/gasmore.svg?raw=1)\n",
        "\n",
        "(Note, this image is produced using the command !otoole viz res excel gas_more.xlsx simple.svg config.yaml)\n",
        "\n",
        "As you can see from the figure above, this model contains multiple gas resources, plus imports, multiple coal resources, plus an import, and a new super-critical coal fired power station.\n",
        "\n",
        "The prices and quantities of the resources are as follows:\n",
        "\n",
        "Resource | Quantity available | Cost of extraction/import\n",
        ":--|--:|--:\n",
        "GasExtraction | 10 | 8.0\n",
        "GasExtraction2 | 15 | 10.0\n",
        "GasExtraction3 | 30 | 11.0\n",
        "GasImport | - | 12.0\n",
        "CoalExtraction | 10 | 4.0\n",
        "CoalExtraction2 | 15 | 5.0\n",
        "CoalExtraction3 | 30 | 5.5\n",
        "CoalImport | - | 12.0\n",
        "\n",
        "These prices and quantities are defined in the `VariableCost` and `TotalTechnologyModelPeriodActivityUpperLimit` sheets respectively.\n",
        "\n",
        "The gas-fired (`NGCC`) and coal-fired (`SCC`) power plants have the following characteristics:\n",
        "\n",
        "Plant | Parameter | Value\n",
        ":--|:--|--:\n",
        "NGCC | CapitalCost | 1100.0\n",
        "SCC | CapitalCost | 1600.0\n",
        "NGCC | FixedCost | 44.0\n",
        "SCC | FixedCost | 56.0\n",
        "NGCC | InputActivityRatio(GAS) | 1.992\n",
        "SCC | InputActivityRatio(COA) | 2.120\n",
        "NGCC | OutputActivityRatio(SEC_EL) | 1.0\n",
        "SCC | OutputActivityRatio(SEC_EL) | 1.0\n",
        "NGCC | ResidualCapacity | 30\n",
        "SCC| ResidualCapacity |30"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='run_s2'></a>\n",
        "Let run this model:"
      ],
      "metadata": {
        "id": "zdDvBCSl93SX"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8ETNNyAg8vm7",
        "tags": []
      },
      "outputs": [],
      "source": [
        "more_results = run_model(\"gas_more\", 1, 60, 2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CWz-7X1i8vm7"
      },
      "outputs": [],
      "source": [
        "more_data = pd.DataFrame(more_results)\n",
        "px.line(more_data, x='quantity', y='value', facet_col='param',\n",
        "        line_shape='hv',\n",
        "        labels={'value': 'Cost'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hI5Ey4lx8vm8"
      },
      "source": [
        "We now have a much more complicated supply cost curve for all commodities. Four distinct steps are evident in the supply cost curve for gas (`GAS`), three/four for coal (`COA`), and seven in  the plot for final electricity (`FEL`). Each of these steps in the gas and coal plots correspond to one of the resources that we defined in this more complicated model. However, the supply cost curve for final electricity (`FEL`) is a function of the combination of the gas and coal cost curves, and depends on the blend of coal and gas used to supply electricity at each level of demand."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zGsIJbR58vm9"
      },
      "outputs": [],
      "source": [
        "production = pd.read_csv('results/ProductionByTechnologyAnnual.csv').groupby(by=['TECHNOLOGY', 'FUEL']).sum()\n",
        "px.bar(production.reset_index(), x='FUEL', y='VALUE', color='TECHNOLOGY',\n",
        "       labels={'VALUE': 'Energy (PJ)'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WIGfKyEb8vm9"
      },
      "source": [
        "### Exercise 4\n",
        "\n",
        "We can confirm the outputs from the model run with maximum demand by viewing the production of the different technologies. In the plot above you can view the quantity of coal and gas extracted or imported, and the quantity of secondary electricity produced by the power plants and final electricity transmitted to meet the demands.\n",
        "\n",
        "**Now, answer quiz question 4 on Canvas**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecyXgx6n8vm9"
      },
      "source": [
        "### Exercise 5\n",
        "\n",
        "To answer this question, you will need to edit the OSeMOSYS parameter that defines the price of resources.  This is the `VariableCost`. Edit the `gas_more.xlsx` model and Run the the cell below to upload the edited model:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_model('gas_more')"
      ],
      "metadata": {
        "id": "ACXm0j_4-Tpt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_8Jq9eQPki1"
      },
      "source": [
        "Now run the model again ([Stage 2](#run_s2)) and generate the plots above."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F-_BUWZbQVcQ"
      },
      "source": [
        "\n",
        "**Now, answer quiz question 5 on Canvas**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5he3ZX2G8vm9"
      },
      "source": [
        "### Exercise 6\n",
        "\n",
        "To answer this question, you will need to edit the OSeMOSYS parameter that defines the efficiency of a technology. These are `InputActivityRatio` and `OutputActivityRatio`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NgRoRMm7QvvJ"
      },
      "source": [
        "Now edit the file upload the new model (running the next cell) and [run again](#run_s2)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_model('gas_more')"
      ],
      "metadata": {
        "id": "1volhvBU_Xdd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y9G6D72jQwlh"
      },
      "source": [
        "**Now, answer quiz question 6 on Canvas**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P4nHhai-8vm-"
      },
      "source": [
        "## Summary of Stage 2\n",
        "\n",
        "In stage 2 we learnt the following:\n",
        "\n",
        "- Our computation of marginal costs using OSeMOSYS holds for a more complicated energy system structure\n",
        "- The short-run marginal cost of electricity is a function of the cost of resources and efficiency of the conversion plants"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kaCUjnTb8vm-"
      },
      "source": [
        "<a name='s3'></a>\n",
        "# Stage 3: A tax on pollution\n",
        "\n",
        "In this stage, we'll now implement a financial penalty for emission of CO2.  \n",
        "\n",
        "### Exercise 7\n",
        "\n",
        "Before proceeding, please **answer quiz question 7 on Canvas**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VVh4DgV28vm-"
      },
      "source": [
        "### Adding an emissions penalty to OSeMOSYS\n",
        "To impose a financial penalty in OSeMOSYS we can edit the `EmissionsPenalty` parameter. Units are \\$/tCO2.\n",
        "\n",
        "You'll need to add a new line to the parameter file like so:\n",
        "\n",
        "    SIMPLICITY,CO2,50\n",
        "    \n",
        "First try a value of $50/tCO2.  You may use the plot below to compare your results with a zero emission price case (from the previous exercise). Edit the model and upload th new `gas_more.xlsx` file running the next cell:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_model('gas_more')"
      ],
      "metadata": {
        "id": "xIcdgMxm_R6I"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "F8PmbSv-8vm-"
      },
      "outputs": [],
      "source": [
        "emission_results = run_model(\"gas_more\", 1, 60, 2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4WPEf6kJ8vm-"
      },
      "outputs": [],
      "source": [
        "emissions_data = pd.DataFrame(emission_results)\n",
        "emissions_data['co2_price'] = 50\n",
        "emit_more_data = more_data.copy()\n",
        "emit_more_data['co2_price'] = 0\n",
        "\n",
        "all_data = pd.concat([emissions_data, emit_more_data], axis=0)\n",
        "px.line(all_data, x='quantity', y='value', color='co2_price', facet_col='param',\n",
        "        line_shape='vh',\n",
        "        labels={'value': 'Cost', 'co2_price': 'CO2 Tax ($/tCO2)'})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_-8Au2QQ8vnF"
      },
      "source": [
        "### Exercise 8\n",
        "Now please go to **quiz question 8 in Canvas**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wJancl6T8vnF"
      },
      "source": [
        "### Exercise 9\n",
        "Please answer **quiz question 9 on Canvas**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o7zoo3vQ8vnF"
      },
      "source": [
        "### Summary of Stage 3\n",
        "In this stage, we have explored the effects of a carbon tax on the supply curves of gas, coal and final electricity. In the exercises on Canvas we have thought about the differences between carbon taxes and emission trading systems. Furthermore, we have analysed how a carbon price affects different fuels and how we can see these effects in the marginal cost curve of electricity."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ybAP80DJ8vnF"
      },
      "source": [
        "<a name='s4'></a>\n",
        "# Stage 4: Renewable electricity\n",
        "In this stage we are adding a solar PV technology and a wind power technology to our model. We are now working with the model in `model/gas_solar_wind.xlsx`. In the following we want to explore and think about the effects that the addition of solar PV and Wind have on the modelled system.\n",
        "![](https://github.com/KTH-dESA/MJ2383/blob/main/img/gassolarwind.svg?raw=1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fe0gGsqp8vnF"
      },
      "source": [
        "The solar PV technology (`SOLPV`) and wind turbine technology (`WIND`) have the following characteristics:\n",
        "\n",
        "Plant | Parameter | Value\n",
        ":--|:--|--:\n",
        "SOLPV | CapitalCost | 1700.0\n",
        "WIND | CapitalCost | 1845.0\n",
        "SOLPV | VariableCost | 2.5\n",
        "WIND| VariableCost |4.17\n",
        "SOLPV | OutputActivityRatio(FEL) | 1.0\n",
        "WIND | OutputActivityRatio(SEC_EL) | 1.0\n",
        "SOLPV | ResidualCapacity | 2\n",
        "WIND| ResidualCapacity |3\n",
        "\n",
        "Solar PV and Wind have the following availability defined in the file `CapacityFactor.csv`:\n",
        "\n",
        "Timeslice | SOLPV | WIND\n",
        ":--|:--|--:\n",
        "ID | 0.3 | 0.25\n",
        "IN | 0.0 | 0.25\n",
        "SD | 0.4 | 0.25\n",
        "SN | 0.0 | 0.25\n",
        "WD | 0.25 | 0.25\n",
        "WN | 0.0 | 0.25\n",
        "\n",
        "You don't have to change anything."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4z6kznnA8vnG"
      },
      "source": [
        "### Exercise 10\n",
        "**Before** running our new model, please think about the effect that solar PV and wind power will have on the marginal cost curves that we looked at in the previous stages. And please **answer quiz question 10 on Canvas**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "44vkdgqD8vnG"
      },
      "outputs": [],
      "source": [
        "results = run_model(\"gas_solar_wind\", 1, 60, 2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "seS3xpHO8vnG"
      },
      "outputs": [],
      "source": [
        "renewable_data = pd.DataFrame(results)\n",
        "px.line(renewable_data, x='quantity', y='value', facet_col='param', range_x=[0,65], range_y=[0,45], line_shape='vh')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UdVJuxA98vnG"
      },
      "outputs": [],
      "source": [
        "production = pd.read_csv('results/ProductionByTechnologyAnnual.csv').groupby(by=['TECHNOLOGY', 'FUEL']).sum()\n",
        "px.bar(production.reset_index(), x='FUEL', y='VALUE', color='TECHNOLOGY')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "doq-MDNL8vnG"
      },
      "source": [
        "### Exercise 11\n",
        "Now please go to **quiz question 11 on Canvas**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jvl-O9c38vnG"
      },
      "source": [
        "### Summary of Stage 4\n",
        "In this stage we explored the effects that renewable power generation technologies have on the marginal cost curve of electricity. We discovered that the introduction of renewable power sources:\n",
        "- does not necessarily push the technologies with the highest emissions out of the market\n",
        "- from a operational point of view a market does not necessarily create a well functioning technology mix, e.g. the combination of coal power with renewable technologies creates operational challenges due to the inertia of coal fired power plants"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sJcTiQMM8vnG"
      },
      "source": [
        "# Extension Tasks\n",
        "\n",
        "If you finish during the lab, you might want to complete one of the following optional tasks:\n",
        "\n",
        "- Return to stage 2 and add a wind or solar technology into the OSeMOSYS model stored in `model/gas_more.xlsx`.  You'll need to edit the following sheets:\n",
        "  - `InputActivityRatio`,\n",
        "  - `OutputActivityRatio`,\n",
        "  - `CapacityFactor`\n",
        "  - `OperationalLife`,\n",
        "  - `ResidualCapacity`,\n",
        "  - `TECHNOLOGY`,\n",
        "  - `FixedCost` and\n",
        "  - `VariableCost`.\n",
        "  \n",
        "  Once you've implemented the renewable technology, observe what happens to the supply cost curve.\n",
        "  \n",
        "  \n",
        "- Return to stage 4 and add an `EmissionsPenalty` to the model, observe what happens to the supply cost curve depending on the hight of penalty you implement.\n",
        "\n",
        "Remember that you can upload the new model by dragging and dropping the file into the model folder, or runnin the next cell:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_model('gas_more')"
      ],
      "metadata": {
        "id": "sCVFlXwXAt8W"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8pHnaLz8vnH"
      },
      "source": [
        "# Summary of lab 3\n",
        "In this lab we have explored the following:\n",
        "- the behaviour of marignal cost curves of different fuels in an OSeMOSYS model in different system settings\n",
        "- the functioning of electricity markets and their sensitivity to price changes\n",
        "- the possibility to extract marginal costs from OSeMOSYS models\n",
        "- the use of a range of parameters to define a power system in OSeMOSYS"
      ]
    }
  ],
  "metadata": {
    "celltoolbar": "Raw Cell Format",
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.16"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}