Development Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a091fc2f-cbd8-456a-a7e1-b21ffa2ea5d7",
   "metadata": {},
   "source": [
    "# Development Notebook\n",
    "\n",
    "Figuring out all these APIs in a notebook first"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "148f2422-7e2d-4ba0-a60b-790e69d7b977",
   "metadata": {},
   "source": [
    "## Imports & Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "493789fe-7906-4542-9ec2-743032b9dabe",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "import json\n",
    "import random\n",
    "import tomllib\n",
    "\n",
    "import pandas as pd\n",
    "import requests\n",
    "import yfinance as yf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d86aa591-65d8-4a15-8f64-5e0d42162f19",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['ALPHA_VANTAGE_KEY', 'GUARDIAN_KEY', 'NASDAQ_KEY'])\n"
     ]
    }
   ],
   "source": [
    "with open(\".secrets\", \"rb\") as secrets_file:\n",
    "    secrets = tomllib.load(secrets_file)\n",
    "print(secrets.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb415f4-4553-4e1c-8669-07baa15706a8",
   "metadata": {},
   "source": [
    "## Grab Market Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e11783a-a017-44a8-8321-0ad71ffc842c",
   "metadata": {},
   "source": [
    "Thank you, Yahoo Finance, for continuting to offer a free API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b952958b-92cd-4a9b-844e-bdceeda2b5e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'maxAge': 86400,\n",
       " 'priceHint': 2,\n",
       " 'previousClose': 14127.282,\n",
       " 'open': 14319.2,\n",
       " 'dayLow': 14243.563,\n",
       " 'dayHigh': 14360.199,\n",
       " 'regularMarketPreviousClose': 14127.282,\n",
       " 'regularMarketOpen': 14319.2,\n",
       " 'regularMarketDayLow': 14243.563,\n",
       " 'regularMarketDayHigh': 14360.199,\n",
       " 'volume': 2389571000,\n",
       " 'regularMarketVolume': 2389571000,\n",
       " 'averageVolume': 4864780983,\n",
       " 'averageVolume10days': 4767984000,\n",
       " 'averageDailyVolume10Day': 4767984000,\n",
       " 'bid': 0.0,\n",
       " 'ask': 0.0,\n",
       " 'bidSize': 0,\n",
       " 'askSize': 0,\n",
       " 'fiftyTwoWeekLow': 10088.83,\n",
       " 'fiftyTwoWeekHigh': 14446.55,\n",
       " 'fiftyDayAverage': 13452.713,\n",
       " 'twoHundredDayAverage': 11882.2,\n",
       " 'currency': 'USD',\n",
       " 'exchange': 'NIM',\n",
       " 'quoteType': 'INDEX',\n",
       " 'symbol': '^IXIC',\n",
       " 'underlyingSymbol': '^IXIC',\n",
       " 'shortName': 'NASDAQ Composite',\n",
       " 'longName': 'NASDAQ Composite',\n",
       " 'firstTradeDateEpochUtc': 34612200,\n",
       " 'timeZoneFullName': 'America/New_York',\n",
       " 'timeZoneShortName': 'EDT',\n",
       " 'uuid': '6b51a47d-53e9-30d4-8a47-289ac3188b0f',\n",
       " 'messageBoardId': 'finmb_INDEXIXIC',\n",
       " 'gmtOffSetMilliseconds': -14400000,\n",
       " 'trailingPegRatio': None}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nasdaq = yf.Ticker(\"^IXIC\")\n",
    "nasdaq.info"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "727214cc-c9b8-4c15-b421-44796cb4bd01",
   "metadata": {},
   "source": [
    "Okay, so this resolved nicely. Let's see what historical data I can fetch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "dc7e277a-575e-4850-8642-4aa8c2f80529",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>Dividends</th>\n",
       "      <th>Stock Splits</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2023-07-24 00:00:00-04:00</th>\n",
       "      <td>14081.629883</td>\n",
       "      <td>14110.150391</td>\n",
       "      <td>13997.129883</td>\n",
       "      <td>14058.870117</td>\n",
       "      <td>4083070000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-25 00:00:00-04:00</th>\n",
       "      <td>14093.240234</td>\n",
       "      <td>14201.910156</td>\n",
       "      <td>14092.519531</td>\n",
       "      <td>14144.559570</td>\n",
       "      <td>3812470000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-26 00:00:00-04:00</th>\n",
       "      <td>14123.519531</td>\n",
       "      <td>14187.349609</td>\n",
       "      <td>14041.950195</td>\n",
       "      <td>14127.280273</td>\n",
       "      <td>4322000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   Open          High           Low  \\\n",
       "Date                                                                  \n",
       "2023-07-24 00:00:00-04:00  14081.629883  14110.150391  13997.129883   \n",
       "2023-07-25 00:00:00-04:00  14093.240234  14201.910156  14092.519531   \n",
       "2023-07-26 00:00:00-04:00  14123.519531  14187.349609  14041.950195   \n",
       "\n",
       "                                  Close      Volume  Dividends  Stock Splits  \n",
       "Date                                                                          \n",
       "2023-07-24 00:00:00-04:00  14058.870117  4083070000        0.0           0.0  \n",
       "2023-07-25 00:00:00-04:00  14144.559570  3812470000        0.0           0.0  \n",
       "2023-07-26 00:00:00-04:00  14127.280273  4322000000        0.0           0.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nasdaq.history(\n",
    "    period=\"1d\", start=dt.date.today() - dt.timedelta(days=3), end=dt.date.today()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48b508b9-82d2-408d-bb58-a5a60d11d525",
   "metadata": {},
   "source": [
    "Nice! That's _literally_ all I need."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a20407d4-ef3a-40a9-bf08-106adbbbbcac",
   "metadata": {},
   "source": [
    "Now to mix it up, there's a few tickers I want to track. Looks like I can download them all in a single call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "056b7870-68a6-4b36-b38b-2c581f5c9009",
   "metadata": {},
   "outputs": [],
   "source": [
    "tickers: dict[str, str] = {\n",
    "    \"NASDAQ\": \"^IXIC\",\n",
    "    \"Dow\": \"^DJI\",\n",
    "    \"S&P 500\": \"^GSPC\",\n",
    "    \"Nikkei\": \"^N225\",\n",
    "    \"FTSE\": \"^FTSE\",\n",
    "    \"Capital One Stock\": \"COF\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "6965751f-cce5-43ef-8097-35c03d7322ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[*********************100%***********************]  6 of 6 completed\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"6\" halign=\"left\">Adj Close</th>\n",
       "      <th colspan=\"4\" halign=\"left\">Close</th>\n",
       "      <th>...</th>\n",
       "      <th colspan=\"4\" halign=\"left\">Open</th>\n",
       "      <th colspan=\"6\" halign=\"left\">Volume</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>COF</th>\n",
       "      <th>^DJI</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>^IXIC</th>\n",
       "      <th>^N225</th>\n",
       "      <th>COF</th>\n",
       "      <th>^DJI</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>...</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>^IXIC</th>\n",
       "      <th>^N225</th>\n",
       "      <th>COF</th>\n",
       "      <th>^DJI</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>^IXIC</th>\n",
       "      <th>^N225</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2023-07-24</th>\n",
       "      <td>117.220001</td>\n",
       "      <td>35411.238281</td>\n",
       "      <td>7678.600098</td>\n",
       "      <td>4554.640137</td>\n",
       "      <td>14058.870117</td>\n",
       "      <td>32700.939453</td>\n",
       "      <td>117.220001</td>\n",
       "      <td>35411.238281</td>\n",
       "      <td>7678.600098</td>\n",
       "      <td>4554.640137</td>\n",
       "      <td>...</td>\n",
       "      <td>7663.700195</td>\n",
       "      <td>4543.390137</td>\n",
       "      <td>14081.629883</td>\n",
       "      <td>32648.140625</td>\n",
       "      <td>3173500</td>\n",
       "      <td>284460000</td>\n",
       "      <td>521826500</td>\n",
       "      <td>3856250000</td>\n",
       "      <td>4083070000</td>\n",
       "      <td>83500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-25</th>\n",
       "      <td>114.510002</td>\n",
       "      <td>35438.070312</td>\n",
       "      <td>7691.799805</td>\n",
       "      <td>4567.459961</td>\n",
       "      <td>14144.559570</td>\n",
       "      <td>32682.509766</td>\n",
       "      <td>114.510002</td>\n",
       "      <td>35438.070312</td>\n",
       "      <td>7691.799805</td>\n",
       "      <td>4567.459961</td>\n",
       "      <td>...</td>\n",
       "      <td>7678.600098</td>\n",
       "      <td>4555.189941</td>\n",
       "      <td>14093.240234</td>\n",
       "      <td>32705.390625</td>\n",
       "      <td>2586100</td>\n",
       "      <td>299530000</td>\n",
       "      <td>444691500</td>\n",
       "      <td>3812470000</td>\n",
       "      <td>3812470000</td>\n",
       "      <td>101000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-26</th>\n",
       "      <td>114.040001</td>\n",
       "      <td>35520.121094</td>\n",
       "      <td>7676.899902</td>\n",
       "      <td>4566.750000</td>\n",
       "      <td>14127.280273</td>\n",
       "      <td>32668.339844</td>\n",
       "      <td>114.040001</td>\n",
       "      <td>35520.121094</td>\n",
       "      <td>7676.899902</td>\n",
       "      <td>4566.750000</td>\n",
       "      <td>...</td>\n",
       "      <td>7691.799805</td>\n",
       "      <td>4558.959961</td>\n",
       "      <td>14123.519531</td>\n",
       "      <td>32704.960938</td>\n",
       "      <td>2666000</td>\n",
       "      <td>346240000</td>\n",
       "      <td>807357200</td>\n",
       "      <td>3990290000</td>\n",
       "      <td>4322000000</td>\n",
       "      <td>85200000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 36 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             Adj Close                                                        \\\n",
       "                   COF          ^DJI        ^FTSE        ^GSPC         ^IXIC   \n",
       "Date                                                                           \n",
       "2023-07-24  117.220001  35411.238281  7678.600098  4554.640137  14058.870117   \n",
       "2023-07-25  114.510002  35438.070312  7691.799805  4567.459961  14144.559570   \n",
       "2023-07-26  114.040001  35520.121094  7676.899902  4566.750000  14127.280273   \n",
       "\n",
       "                               Close                                          \\\n",
       "                   ^N225         COF          ^DJI        ^FTSE        ^GSPC   \n",
       "Date                                                                           \n",
       "2023-07-24  32700.939453  117.220001  35411.238281  7678.600098  4554.640137   \n",
       "2023-07-25  32682.509766  114.510002  35438.070312  7691.799805  4567.459961   \n",
       "2023-07-26  32668.339844  114.040001  35520.121094  7676.899902  4566.750000   \n",
       "\n",
       "            ...         Open                                           \\\n",
       "            ...        ^FTSE        ^GSPC         ^IXIC         ^N225   \n",
       "Date        ...                                                         \n",
       "2023-07-24  ...  7663.700195  4543.390137  14081.629883  32648.140625   \n",
       "2023-07-25  ...  7678.600098  4555.189941  14093.240234  32705.390625   \n",
       "2023-07-26  ...  7691.799805  4558.959961  14123.519531  32704.960938   \n",
       "\n",
       "             Volume                                                           \n",
       "                COF       ^DJI      ^FTSE       ^GSPC       ^IXIC      ^N225  \n",
       "Date                                                                          \n",
       "2023-07-24  3173500  284460000  521826500  3856250000  4083070000   83500000  \n",
       "2023-07-25  2586100  299530000  444691500  3812470000  3812470000  101000000  \n",
       "2023-07-26  2666000  346240000  807357200  3990290000  4322000000   85200000  \n",
       "\n",
       "[3 rows x 36 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stonks = yf.download(\n",
    "    \" \".join(tickers.values()),\n",
    "    start=dt.date.today() - dt.timedelta(days=3),\n",
    "    end=dt.date.today(),\n",
    ")\n",
    "stonks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b507089-b9c9-4481-9d50-4f4762d0e6b2",
   "metadata": {},
   "source": [
    "And really I want to boil this down to a single data point for each."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "b581c211-c2e9-44d4-8c8d-8a560b7f02f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COF</th>\n",
       "      <th>^DJI</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>^IXIC</th>\n",
       "      <th>^N225</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2023-07-24</th>\n",
       "      <td>1.462826</td>\n",
       "      <td>0.512192</td>\n",
       "      <td>0.194422</td>\n",
       "      <td>0.247612</td>\n",
       "      <td>-0.161627</td>\n",
       "      <td>0.161721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-25</th>\n",
       "      <td>-2.411790</td>\n",
       "      <td>0.046813</td>\n",
       "      <td>0.171903</td>\n",
       "      <td>0.269364</td>\n",
       "      <td>0.364141</td>\n",
       "      <td>-0.069961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-26</th>\n",
       "      <td>-0.886491</td>\n",
       "      <td>0.492652</td>\n",
       "      <td>-0.193712</td>\n",
       "      <td>0.170873</td>\n",
       "      <td>0.026628</td>\n",
       "      <td>-0.111974</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 COF      ^DJI     ^FTSE     ^GSPC     ^IXIC     ^N225\n",
       "Date                                                                  \n",
       "2023-07-24  1.462826  0.512192  0.194422  0.247612 -0.161627  0.161721\n",
       "2023-07-25 -2.411790  0.046813  0.171903  0.269364  0.364141 -0.069961\n",
       "2023-07-26 -0.886491  0.492652 -0.193712  0.170873  0.026628 -0.111974"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "100 * (stonks.Close - stonks.Open) / stonks.Open"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f8ccb77-0610-4772-94a0-bf1008eee17e",
   "metadata": {},
   "source": [
    "And honestly I want it even more basic than that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "77ea55ef-39ac-41d4-87e2-797f08a10ae8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COF</th>\n",
       "      <th>^DJI</th>\n",
       "      <th>^FTSE</th>\n",
       "      <th>^GSPC</th>\n",
       "      <th>^IXIC</th>\n",
       "      <th>^N225</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2023-07-24</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-25</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-07-26</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              COF  ^DJI  ^FTSE  ^GSPC  ^IXIC  ^N225\n",
       "Date                                               \n",
       "2023-07-24   True  True   True   True  False   True\n",
       "2023-07-25  False  True   True   True   True  False\n",
       "2023-07-26  False  True  False   True   True  False"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(stonks.Close - stonks.Open) > 0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6a05780-de50-4bd9-ade5-e7c1515c8d8d",
   "metadata": {},
   "source": [
    "## News Stories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc80945-2e27-4f73-b3c5-28415cc90aed",
   "metadata": {},
   "source": [
    "Yet another reason to love The Guardian."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "8dbb016a-2dc1-46e7-94c8-85690356d758",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Response [200]>\n"
     ]
    }
   ],
   "source": [
    "result = requests.get(\n",
    "    \"https://content.guardianapis.com/sections\",\n",
    "    params={\n",
    "        \"api-key\": secrets[\"GUARDIAN_KEY\"],\n",
    "        \"from-date\": dt.date.today() - dt.timedelta(days=3),\n",
    "        \"to-date\": dt.date.today(),\n",
    "    },\n",
    ")\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6916ae58-7b02-4efb-97ef-4bbad8f9d37d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['About',\n",
       " 'Animals farmed',\n",
       " 'Art and design',\n",
       " 'Australia news',\n",
       " 'Better Business',\n",
       " 'Books',\n",
       " 'Business',\n",
       " 'Business to business',\n",
       " 'Cardiff',\n",
       " \"Children's books\",\n",
       " 'Cities',\n",
       " 'Opinion',\n",
       " 'Community',\n",
       " 'Crosswords',\n",
       " 'Culture',\n",
       " 'Culture Network',\n",
       " 'Culture professionals network',\n",
       " 'Edinburgh',\n",
       " 'Education',\n",
       " 'Guardian Enterprise Network',\n",
       " 'Environment',\n",
       " 'Extra',\n",
       " 'Fashion',\n",
       " 'Film',\n",
       " 'Food',\n",
       " 'Football',\n",
       " 'Games',\n",
       " 'Global development',\n",
       " 'Global Development Professionals Network',\n",
       " 'Guardian Government Computing',\n",
       " 'Guardian Foundation',\n",
       " 'Guardian Professional',\n",
       " 'Healthcare Professionals Network',\n",
       " 'Help',\n",
       " 'Higher Education Network',\n",
       " 'Housing Network',\n",
       " 'Inequality',\n",
       " 'Info',\n",
       " 'Jobs',\n",
       " 'Katine',\n",
       " 'Law',\n",
       " 'Leeds',\n",
       " 'Life and style',\n",
       " 'Local',\n",
       " 'Local Leaders Network',\n",
       " 'Media',\n",
       " 'Media Network',\n",
       " 'Membership',\n",
       " 'Money',\n",
       " 'Music',\n",
       " 'News',\n",
       " 'Politics',\n",
       " 'Public Leaders Network',\n",
       " 'Science',\n",
       " 'Search',\n",
       " 'Guardian Small Business Network',\n",
       " 'Social Care Network',\n",
       " 'Social Enterprise Network',\n",
       " 'Society',\n",
       " 'Society Professionals',\n",
       " 'Sport',\n",
       " 'Stage',\n",
       " 'Teacher Network',\n",
       " 'Technology',\n",
       " 'From the Guardian',\n",
       " 'From the Observer',\n",
       " 'Travel',\n",
       " 'Guardian holiday offers',\n",
       " 'Television & radio',\n",
       " 'UK news',\n",
       " 'US news',\n",
       " 'Voluntary Sector Network',\n",
       " 'Weather',\n",
       " 'Women in Leadership',\n",
       " 'Working in development',\n",
       " 'World news']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[section[\"webTitle\"] for section in json.loads(result.text)[\"response\"][\"results\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "526f76ba-b716-48ff-98d4-816cdea983ce",
   "metadata": {},
   "source": [
    "So ideally we want to filter out finance-related topics and a few other sections that probably won't map well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "6c30ff4d-9b65-4c37-8ed5-4febb8be6b93",
   "metadata": {},
   "outputs": [],
   "source": [
    "excluded_sections = [\n",
    "    \"About\",\n",
    "    \"Better Business\",\n",
    "    \"Business\",\n",
    "    \"Business to business\",\n",
    "    \"Opinion\",\n",
    "    \"Community\",\n",
    "    \"Crosswords\",\n",
    "    \"Global development\",\n",
    "    \"Help\",\n",
    "    \"Inequality\",\n",
    "    \"Info\",\n",
    "    \"Jobs\",\n",
    "    \"Membership\",\n",
    "    \"Money\",\n",
    "    \"News\",\n",
    "    \"Politics\",\n",
    "    \"Search\",\n",
    "    \"From the Guardian\",\n",
    "    \"From the Observer\",\n",
    "    \"Guardian holiday offers\",\n",
    "    \"World news\",\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "9430428c-2681-48b1-aa0a-892f3b47eb7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Response [200]>\n"
     ]
    }
   ],
   "source": [
    "result = requests.get(\n",
    "    \"https://content.guardianapis.com/search\",\n",
    "    params={\n",
    "        \"api-key\": secrets[\"GUARDIAN_KEY\"],\n",
    "        \"from-date\": dt.date.today() - dt.timedelta(days=3),\n",
    "        \"to-date\": dt.date.today(),\n",
    "        \"page-size\": 50,\n",
    "        \"section\": \",\".join([f\"-{section}\" for section in excluded_sections]),\n",
    "    },\n",
    ")\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "fe9284b7-77b3-4f7e-aa92-4e7f9efa4fd6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'sport/live/2023/jul/27/england-v-australia-ashes-fifth-test-day-one-live-scores-updates-results-aus-vs-eng-cricket-the-oval',\n",
       " 'type': 'liveblog',\n",
       " 'sectionId': 'sport',\n",
       " 'sectionName': 'Sport',\n",
       " 'webPublicationDate': '2023-07-27T17:45:43Z',\n",
       " 'webTitle': 'England v Australia: Ashes fifth Test, day one – live reaction',\n",
       " 'webUrl': 'https://www.theguardian.com/sport/live/2023/jul/27/england-v-australia-ashes-fifth-test-day-one-live-scores-updates-results-aus-vs-eng-cricket-the-oval',\n",
       " 'apiUrl': 'https://content.guardianapis.com/sport/live/2023/jul/27/england-v-australia-ashes-fifth-test-day-one-live-scores-updates-results-aus-vs-eng-cricket-the-oval',\n",
       " 'isHosted': False,\n",
       " 'pillarId': 'pillar/sport',\n",
       " 'pillarName': 'Sport'}"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.loads(result.text)[\"response\"][\"results\"][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "962a2989-792b-4c5d-9aa9-133f04378940",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['England v Australia: Ashes fifth Test, day one – live reaction',\n",
       " 'Trump says lawyers were given no indication of looming indictment from DoJ – live',\n",
       " 'Quashing of Andrew Malkinson’s rape conviction confirms failings of criminal review watchdog',\n",
       " 'The Labour party is walking a fine line on trans rights | Letters',\n",
       " 'Home Office is racist for blocking Siyabonga Twala from returning | Letter',\n",
       " 'Disabled people are receiving degrading treatment in an overstretched NHS | Letter',\n",
       " 'A short-lived guide to saving the Earth',\n",
       " 'Sunak under pressure to block ex-Ukip deputy from potential Tory candidacy',\n",
       " 'The Guardian view on the freeing of Andy Malkinson: a case for reform | Editorial',\n",
       " 'The Guardian view on levelling up: widening regional pay gaps expose Conservative failure | Editorial',\n",
       " 'Muslim leaders decry ‘double standard’ of Farage bank account closure furore',\n",
       " 'First 50 people coming to Bibby Stockholm asylum barge despite safety worries',\n",
       " 'Russia-Ukraine war live: Putin says Ukrainian attacks have intensified; EU says it can’t fund Ukraine grain transport',\n",
       " 'Norwegian woman claims record time for climbing world’s 14 highest peaks',\n",
       " 'Jordan Henderson no longer an LGBTQ+ ally after Saudi move, says Hitzlsperger',\n",
       " 'Bereaved mother urges Women’s Super League to reject gambling partnerships',\n",
       " 'Matildas face date with destiny after fluffing their lines against Nigeria | Kieran Pender',\n",
       " 'British Gas profits bonanza tells you energy price cap has failed | Nils Pratley',\n",
       " 'Ashes diary: Stokes scorns idea of rest as Gregg Wallace enjoys cricket feast',\n",
       " 'Ukraine calls for disqualified fencer to be reinstated after anti-Russia protest',\n",
       " 'Republicans plan Zuckerberg contempt vote in ‘censorship’ investigation',\n",
       " 'Ricarda Bauernfeind wins brutal Tour de France Femmes stage after dropouts',\n",
       " 'County cricket: Somerset v Surrey, Essex beat Hampshire and more – live',\n",
       " 'Jordan Henderson deserves to take the heat for Saudi Arabia sellout',\n",
       " 'George Russell calls for Belgian Grand Prix to be cancelled if unsafe for drivers',\n",
       " 'Coutts CEO stands down in Farage bank account row, as Sunak fails to back NatWest chairman Davies – as it happened',\n",
       " '‘It’s exactly as they’d have done it in the 1910s’: how Barbenheimer is leading the anti-CGI backlash',\n",
       " 'Explainer: Niger a linchpin for stability in Africa’s ‘coup belt’',\n",
       " 'Tour de France Femmes: Ricarda Bauernfeind wins stage five – as it happened',\n",
       " 'A Black prosecutor was elected in Georgia – so white Republicans made their own district',\n",
       " 'Teenage Mutant Ninja Turtles: Mutant Mayhem review – evergreen superheroes rise up from the drains',\n",
       " 'The US at this World Cup are young, talented … and running out of time to peak',\n",
       " 'Even in the heat of war, there must be dialogue. To save lives, a new grain deal with Putin must be struck | Simon Jenkins',\n",
       " ' Mitch McConnell fell earlier this month, before freezing mid-sentence this week',\n",
       " '‘She trembled with the truths she had to tell’: Sinéad O’Connor by friends, fans and collaborators',\n",
       " 'EU opens antitrust inquiry into Microsoft’s Teams software',\n",
       " 'ECB urged to pause interest rate rises amid recession fears',\n",
       " 'Khan dismisses Sunak’s attack on his housebuilding record in London as ‘desperate nonsense’ – as it happened',\n",
       " 'Black fraternity moves Florida event over DeSantis’s ‘assault’ on minorities',\n",
       " 'Twenty renters fighting over each flat in Britain? I worry I’ll never find a place to call home | Anonymous',\n",
       " 'Brazil: descendants of Africans who escaped slavery gain census recognition',\n",
       " '‘We practically skipped out of the cinema on a high’: readers on going to watch Barbie',\n",
       " 'ITV warns it is in ‘worst ad recession since financial crisis’',\n",
       " 'Netball World Cup: challengers close gap on Australia and New Zealand',\n",
       " '‘Birmingham could pull it off’: city considers hosting Commonwealth Games again',\n",
       " ' El Salvador clears way for mass trials as crackdown on gangs ramps up',\n",
       " 'Nadine Dorries urged to quit as MP by council in her constituency',\n",
       " 'Where are the ‘violet hues’ and ‘bath of heat’?: Australian scientists review what Oppenheimer gets wrong',\n",
       " 'Thousands of vapes seized in Melbourne raid as police send ‘clear message’ to stores',\n",
       " 'Sweeney Todd review – a solid, haunting take on Sondheim’s demon barber']"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[article[\"webTitle\"] for article in json.loads(result.text)[\"response\"][\"results\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e438f5-55eb-488b-b567-dc5e936ddbea",
   "metadata": {},
   "source": [
    "Okay, some cleanup to do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "e3b9159a-4c1a-4483-a93c-0f3894aceecd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Ashes fifth Test, day one',\n",
       " 'Trump says lawyers were given no indication of looming indictment from DoJ',\n",
       " 'Quashing of Andrew Malkinson’s rape conviction confirms failings of criminal review watchdog',\n",
       " 'The Labour party is walking a fine line on trans rights',\n",
       " 'Home Office is racist for blocking Siyabonga Twala from returning',\n",
       " 'Disabled people are receiving degrading treatment in an overstretched NHS',\n",
       " 'A short-lived guide to saving the Earth',\n",
       " 'Sunak under pressure to block ex-Ukip deputy from potential Tory candidacy',\n",
       " 'a case for reform',\n",
       " 'widening regional pay gaps expose Conservative failure',\n",
       " 'Muslim leaders decry ‘double standard’ of Farage bank account closure furore',\n",
       " 'First 50 people coming to Bibby Stockholm asylum barge despite safety worries',\n",
       " 'Putin says Ukrainian attacks have intensified',\n",
       " 'Norwegian woman claims record time for climbing world’s 14 highest peaks',\n",
       " 'Jordan Henderson no longer an LGBTQ+ ally after Saudi move, says Hitzlsperger',\n",
       " 'Bereaved mother urges Women’s Super League to reject gambling partnerships',\n",
       " 'Matildas face date with destiny after fluffing their lines against Nigeria',\n",
       " 'British Gas profits bonanza tells you energy price cap has failed',\n",
       " 'Stokes scorns idea of rest as Gregg Wallace enjoys cricket feast',\n",
       " 'Ukraine calls for disqualified fencer to be reinstated after anti-Russia protest',\n",
       " 'Republicans plan Zuckerberg contempt vote in ‘censorship’ investigation',\n",
       " 'Ricarda Bauernfeind wins brutal Tour de France Femmes stage after dropouts',\n",
       " 'Somerset v Surrey, Essex beat Hampshire and more',\n",
       " 'Jordan Henderson deserves to take the heat for Saudi Arabia sellout',\n",
       " 'George Russell calls for Belgian Grand Prix to be cancelled if unsafe for drivers',\n",
       " 'Coutts CEO stands down in Farage bank account row, as Sunak fails to back NatWest chairman Davies',\n",
       " 'how Barbenheimer is leading the anti-CGI backlash',\n",
       " 'Niger a linchpin for stability in Africa’s ‘coup belt’',\n",
       " 'Ricarda Bauernfeind wins stage five',\n",
       " 'A Black prosecutor was elected in Georgia',\n",
       " 'Mutant Mayhem review',\n",
       " 'The US at this World Cup are young, talented … and running out of time to peak',\n",
       " 'Even in the heat of war, there must be dialogue. To save lives, a new grain deal with Putin must be struck',\n",
       " 'Mitch McConnell fell earlier this month, before freezing mid-sentence this week',\n",
       " 'Sinéad O’Connor by friends, fans and collaborators',\n",
       " 'EU opens antitrust inquiry into Microsoft’s Teams software',\n",
       " 'ECB urged to pause interest rate rises amid recession fears',\n",
       " 'Khan dismisses Sunak’s attack on his housebuilding record in London as ‘desperate nonsense’',\n",
       " 'Black fraternity moves Florida event over DeSantis’s ‘assault’ on minorities',\n",
       " 'Twenty renters fighting over each flat in Britain? I worry I’ll never find a place to call home',\n",
       " 'descendants of Africans who escaped slavery gain census recognition',\n",
       " 'readers on going to watch Barbie',\n",
       " 'ITV warns it is in ‘worst ad recession since financial crisis’',\n",
       " 'challengers close gap on Australia and New Zealand',\n",
       " 'city considers hosting Commonwealth Games again',\n",
       " 'El Salvador clears way for mass trials as crackdown on gangs ramps up',\n",
       " 'Nadine Dorries urged to quit as MP by council in her constituency',\n",
       " 'Australian scientists review what Oppenheimer gets wrong',\n",
       " 'Thousands of vapes seized in Melbourne raid as police send ‘clear message’ to stores',\n",
       " 'Sweeney Todd review']"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def clean_headline(headline: str) -> str:\n",
    "    \"\"\"Get to the relevant \"sound byte\" of a headline\"\"\"\n",
    "    return headline.split(\"–\")[0].split(\":\")[-1].split(\";\")[0].split(\"|\")[0].strip()\n",
    "\n",
    "\n",
    "[\n",
    "    clean_headline(article[\"webTitle\"])\n",
    "    for article in json.loads(result.text)[\"response\"][\"results\"]\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8805a22d-6337-4272-8aff-d58c4aeed1ef",
   "metadata": {},
   "source": [
    "I think I can work with this. Let's expand the sample size."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec394d20-8748-4619-b18d-8e4f41b12b67",
   "metadata": {},
   "source": [
    "Now, for a given article, can I also get the lede?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "80718ddd-fc28-4b12-b00f-e3253ab4f904",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'status': 'ok',\n",
       " 'userTier': 'developer',\n",
       " 'total': 1,\n",
       " 'content': {'id': 'society/2023/jul/27/disabled-people-are-receiving-degrading-treatment-in-an-overstretched-nhs',\n",
       "  'type': 'article',\n",
       "  'sectionId': 'society',\n",
       "  'sectionName': 'Society',\n",
       "  'webPublicationDate': '2023-07-27T17:34:59Z',\n",
       "  'webTitle': 'Disabled people are receiving degrading treatment in an overstretched NHS | Letter',\n",
       "  'webUrl': 'https://www.theguardian.com/society/2023/jul/27/disabled-people-are-receiving-degrading-treatment-in-an-overstretched-nhs',\n",
       "  'apiUrl': 'https://content.guardianapis.com/society/2023/jul/27/disabled-people-are-receiving-degrading-treatment-in-an-overstretched-nhs',\n",
       "  'fields': {'body': '<p>It’s deeply concerning to hear that frontline NHS staff are feeling “moral distress” from having too little time to spend with their patients (<a href=\"https://www.theguardian.com/society/2023/jul/24/most-nhs-staff-say-they-dont-have-enough-time-to-spend-with-patients\" title=\"\">Most NHS staff say they don’t have enough time to spend with patients, 24 July</a>). It creates a worrying picture for people with a learning disability, who often need extra time for appointments and already struggle to access basic healthcare.</p> <p>Our helpline regularly receives calls from families seriously worried about quality of care and feeling they need to be there to advocate for their loved one. We hear stories of people being left in incontinence pads rather than being supported to go to the toilet, or having medication and care plans changed without clear explanation. We know that people on specialised diets have been fed the wrong food, sometimes with tragic consequences that could have been avoidable if time was spent reading their hospital notes.</p> <p>This treatment is degrading and can have a long-lasting impact on people beyond their stay in hospital, undoing years of hard work gaining basic life skills to live independently.</p> <p>It’s essential that all appointments for people with learning disabilities include reasonable adjustments that are person-centred and respond to an individual’s particular needs. It’s also a legal right.</p> <p>The new NHS workforce strategy includes some positive changes for patients with learning disabilities, including increasing the number of learning disability nurses and developing NHS training programmes to enable health and care practitioners to expertly treat people with a learning disability across all aspects of medicine. This will hugely improve the quality of care people receive, but we are years away from this having an impact on people’s lives and it’s not going to fix the problem of our NHS staff feeling overstretched and time poor. <br incopy-break-type=\"webbreak\"><strong>Dan Scorer</strong> <br incopy-break-type=\"webbreak\"><em>Head of policy, public affairs, information and advice</em><em>, </em><em>Mencap</em></p> <p><strong><em>• Have an opinion on anything you’ve read in the Guardian today? Please </em></strong><a href=\"mailto:guardian.letters@theguardian.com\" title=\"\"><strong><em>email</em></strong></a><strong><em> us your letter and it will be considered for publication in our </em></strong><a href=\"https://www.theguardian.com/tone/letters\" title=\"\"><strong><em>letters</em></strong></a><strong><em> section.</em></strong></p>'},\n",
       "  'isHosted': False,\n",
       "  'pillarId': 'pillar/news',\n",
       "  'pillarName': 'News'}}"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.loads(\n",
    "    requests.get(\n",
    "        json.loads(result.text)[\"response\"][\"results\"][5][\"apiUrl\"],\n",
    "        params={\n",
    "            \"api-key\": secrets[\"GUARDIAN_KEY\"],\n",
    "            \"show-fields\": [\"trailText\", \"headline\", \"body\"],\n",
    "        },\n",
    "    ).text\n",
    ")[\"response\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "fe8c7fae-38db-46b0-a527-50d4f3df44de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'body': '<p>It’s deeply concerning to hear that frontline NHS staff are feeling “moral distress” from having too little time to spend with their patients (<a href=\"https://www.theguardian.com/society/2023/jul/24/most-nhs-staff-say-they-dont-have-enough-time-to-spend-with-patients\" title=\"\">Most NHS staff say they don’t have enough time to spend with patients, 24 July</a>). It creates a worrying picture for people with a learning disability, who often need extra time for appointments and already struggle to access basic healthcare.</p> <p>Our helpline regularly receives calls from families seriously worried about quality of care and feeling they need to be there to advocate for their loved one. We hear stories of people being left in incontinence pads rather than being supported to go to the toilet, or having medication and care plans changed without clear explanation. We know that people on specialised diets have been fed the wrong food, sometimes with tragic consequences that could have been avoidable if time was spent reading their hospital notes.</p> <p>This treatment is degrading and can have a long-lasting impact on people beyond their stay in hospital, undoing years of hard work gaining basic life skills to live independently.</p> <p>It’s essential that all appointments for people with learning disabilities include reasonable adjustments that are person-centred and respond to an individual’s particular needs. It’s also a legal right.</p> <p>The new NHS workforce strategy includes some positive changes for patients with learning disabilities, including increasing the number of learning disability nurses and developing NHS training programmes to enable health and care practitioners to expertly treat people with a learning disability across all aspects of medicine. This will hugely improve the quality of care people receive, but we are years away from this having an impact on people’s lives and it’s not going to fix the problem of our NHS staff feeling overstretched and time poor. <br incopy-break-type=\"webbreak\"><strong>Dan Scorer</strong> <br incopy-break-type=\"webbreak\"><em>Head of policy, public affairs, information and advice</em><em>, </em><em>Mencap</em></p> <p><strong><em>• Have an opinion on anything you’ve read in the Guardian today? Please </em></strong><a href=\"mailto:guardian.letters@theguardian.com\" title=\"\"><strong><em>email</em></strong></a><strong><em> us your letter and it will be considered for publication in our </em></strong><a href=\"https://www.theguardian.com/tone/letters\" title=\"\"><strong><em>letters</em></strong></a><strong><em> section.</em></strong></p>'}"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_[\"content\"][\"fields\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "ec5d4cd2-fbd2-4917-9202-48f1e0096114",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['body'])"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfa3fc05-f643-4f35-919e-c537d700cff9",
   "metadata": {},
   "source": [
    "Feh. It'll do, though."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867137f8-2225-4e97-b91a-29175d4fd4f8",
   "metadata": {},
   "source": [
    "## Combine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "5cd35cc5-84d4-4ac7-88d5-b8dd943097b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "up_words: list[str] = [\n",
    "    \"soars\",\n",
    "    \"skyrockets\",\n",
    "    \"captapults\",\n",
    "    \"zooms\",\n",
    "    \"jumps\",\n",
    "    \"shoots up\",\n",
    "]\n",
    "\n",
    "down_words: list[str] = [\n",
    "    \"plummets\",\n",
    "    \"tanks\",\n",
    "    \"in free fall\",\n",
    "    \"dives\",\n",
    "    # \"crashes\",  # let's not go that far\n",
    "    \"plunges\",\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "6fec5029-821f-4492-a65d-8d90d5a0cedf",
   "metadata": {},
   "outputs": [],
   "source": [
    "random.seed()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "93a5cda3-72a0-4312-b530-ada5c15aa73d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Nikkei dives as The US at this World Cup are young, talented … and running out of time to peak\n"
     ]
    }
   ],
   "source": [
    "indicator, symbol = random.choice(list(tickers.items()))\n",
    "is_up = (stonks.Close - stonks.Open).at[stonks.index[-1], symbol] > 0\n",
    "\n",
    "word = random.choice(up_words if is_up else down_words)\n",
    "\n",
    "article = random.choice(json.loads(result.text)[\"response\"][\"results\"])\n",
    "base_headline = clean_headline(article[\"webTitle\"])\n",
    "\n",
    "print(\" \".join([indicator, word, \"as\", base_headline]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86cc1487-0760-431f-b3ac-912a8db887e2",
   "metadata": {},
   "source": [
    "Omg. That's epic. Now gimme the lede."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "2c9120e6-4068-4d70-97e0-bd9907f79ea6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When the United States won the 2019 Women’s World Cup, they did so with a team certain of their identity, one that pressured opponents into submission early in each match. The Americans scored in the 12th minute or earlier in each of their first six games of that campaign. A defining characteristic of the USA in that era was their high press and counter-press after losing possession. It was suffocating and relentless – and it forced some of the best teams in the world to panic. \n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    json.loads(\n",
    "        requests.get(\n",
    "            article[\"apiUrl\"],\n",
    "            params={\"api-key\": secrets[\"GUARDIAN_KEY\"], \"show-fields\": [\"body\"]},\n",
    "        ).text\n",
    "    )[\"response\"][\"content\"][\"fields\"][\"body\"].split(\"</p>\")[0][3:]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e465edef-db57-4406-a055-ef503fb152ef",
   "metadata": {},
   "source": [
    "And finally a link tot he content:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "6d649cb8-5e5e-4679-b979-2d2f01a40792",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://www.theguardian.com/football/2023/jul/27/uswnt-usa-womens-world-cup-soccer-netherlands'"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "article[\"webUrl\"]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}