13-BotService-API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "73a710b9-e823-459f-a5db-d66165f05b03",
   "metadata": {},
   "source": [
    "# Using the Bot Service API programmatically"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "570cac31-cb62-4a94-90fc-c7da48e856a3",
   "metadata": {},
   "source": [
    "In the previous notebook, we developed a Backend Web API utilizing the Bot Framework, which is hosted on the Azure Web App service.\n",
    "\n",
    "Additionally, we crafted a demonstration Front End using Streamlit, incorporating a JavaScript snippet to display an engaging chat window. This chat interface can be seamlessly embedded and customized as needed.\n",
    "\n",
    "In this notebook, we will programmatically interact with this API using pure Python code through the Direct Line Channel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e6404ca7-5a5f-4b66-b341-211a394810ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import aiohttp\n",
    "import asyncio\n",
    "import time \n",
    "import requests\n",
    "import datetime\n",
    "import pytz\n",
    "\n",
    "import nest_asyncio\n",
    "nest_asyncio.apply()  # this is only needed for jupyter notebooks\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")\n",
    "\n",
    "# Declare Bot Service variables\n",
    "\n",
    "base_url = os.environ[\"BOT_DIRECT_CHANNEL_ENDPOINT\"]\n",
    "bot_id = os.environ[\"BOT_ID\"]\n",
    "direct_line_secret = os.environ[\"BOT_SERVICE_DIRECT_LINE_SECRET\"]\n",
    "\n",
    "timeout = 25"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab42e7b7-3412-49b7-a090-8b52b5d6851a",
   "metadata": {},
   "source": [
    "Regarding the Bot Service Direct Line channel, there are two primary endpoints of interest:\n",
    "\n",
    "- `{base_url}/conversations`: Initiates the conversation and provides the conversation ID.\n",
    "- `{base_url}/conversations/{conversation_id}/activities`: Asynchronously returns every activity occurring within the bot.\n",
    "\n",
    "The fundamental workflow involves:\n",
    "1) Initiating a new conversation by utilizing the `/conversations` endpoint.\n",
    "2) Sending messages through the `/conversations/{conversation_id}/activities` endpoint.\n",
    "3) Periodically polling the `/conversations/{conversation_id}/activities` endpoint to retrieve responses, errors, and other relevant activities.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9c1c82bb-206e-4023-be95-d79b7ccfb71b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Converstion id: 9wah6oavbyqHdVAOR5hBEx-au\n",
      "CPU times: user 130 ms, sys: 4.95 ms, total: 135 ms\n",
      "Wall time: 11.6 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# Simple workflow\n",
    "start_conversation_url = f\"{base_url}/conversations\"\n",
    "\n",
    "# 1- Start a conversation\n",
    "headers = {\"Authorization\": f\"Bearer {direct_line_secret}\"}\n",
    "response = requests.post(start_conversation_url, headers=headers)\n",
    "conversation_id = response.json()[\"conversationId\"]\n",
    "print('Converstion id:', conversation_id)\n",
    "\n",
    "get_activities_url = f\"{base_url}/conversations/{conversation_id}/activities\"\n",
    "\n",
    "\n",
    "# 2 - Send a message to the bot\n",
    "send_message_url = f\"{base_url}/conversations/{conversation_id}/activities\"\n",
    "message = {\n",
    "    \"type\": \"message\",\n",
    "    \"from\": {\"id\": \"user\"},\n",
    "    \"text\": \"what is CLP?\"\n",
    "}\n",
    "\n",
    "response = requests.post(send_message_url, headers=headers, json=message)\n",
    "\n",
    "# 3 - Wait a bit and Get Activities\n",
    "time.sleep(5)\n",
    "response = requests.get(get_activities_url, headers=headers)\n",
    "activities = response.json()[\"activities\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1ebc13c0-10b1-42c0-9280-89b237634825",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'message',\n",
       "  'id': '9wah6oavbyqHdVAOR5hBEx-au|0000000',\n",
       "  'timestamp': '2024-07-10T11:11:04.50885Z',\n",
       "  'channelId': 'directline',\n",
       "  'from': {'id': 'BotId-zf4fwhz3gdn64', 'name': 'BotId-zf4fwhz3gdn64'},\n",
       "  'conversation': {'id': '9wah6oavbyqHdVAOR5hBEx-au'},\n",
       "  'text': '\\nHello and welcome! 👋\\n\\nMy name is Jarvis, a smart virtual assistant designed to assist you.\\nHere\\'s how you can interact with me:\\n\\nI have various plugins and tools at my disposal to answer your questions effectively. Here are the available options:\\n\\n1. 🌐 **bing**: This tool allows me to access the internet and provide current information from the web.\\n\\n2. 💡 **chatgpt**: With this tool, I can draw upon my own knowledge based on the data I was trained on. Please note that my training data goes up until 2021.\\n\\n3. 🔍 **docsearch**: This tool allows me to search a specialized search engine index. It includes 10,000 ArXiv computer science documents from 2020-2021 and 90,000 Covid research articles from the same years.\\n\\n4. 📖 **booksearch**: This tool allows me to search on 5 specific books: Rich Dad Poor Dad, Made to Stick, Azure Cognitive Search Documentation, Fundamentals of Physics and Boundaries.\\n\\n5. 📊 **sqlsearch**: By utilizing this tool, I can access a SQL database containing information about Covid cases, deaths, and hospitalizations in 2020-2021.\\n\\nFrom all of my sources, I will provide the necessary information and also mention the sources I used to derive the answer. This way, you can have transparency about the origins of the information and understand how I arrived at the response.\\n\\nTo make the most of my capabilities, please mention the specific tool you\\'d like me to use when asking your question. Here\\'s an example:\\n\\n```\\nbing, who is the daughter of the President of India?\\nchatgpt, how can I read a remote file from a URL using pandas?\\ndocsearch, Does chloroquine really works against covid?\\nbooksearch, tell me the legend of the stolen kidney in the book \"Made To Stick\"\\nsqlsearch, how many people died on the West Coast in 2020?\\n```\\n\\nFeel free to ask any question and specify the tool you\\'d like me to utilize. I\\'m here to assist you!\\n\\n---\\n',\n",
       "  'inputHint': 'acceptingInput',\n",
       "  'replyToId': '8V0qwdc5wOf'},\n",
       " {'type': 'message',\n",
       "  'id': '9wah6oavbyqHdVAOR5hBEx-au|0000001',\n",
       "  'timestamp': '2024-07-10T11:11:03.6242788Z',\n",
       "  'serviceUrl': 'https://directline.botframework.com/',\n",
       "  'channelId': 'directline',\n",
       "  'from': {'id': 'user'},\n",
       "  'conversation': {'id': '9wah6oavbyqHdVAOR5hBEx-au'},\n",
       "  'text': 'what is CLP?'},\n",
       " {'type': 'message',\n",
       "  'id': '9wah6oavbyqHdVAOR5hBEx-au|0000002',\n",
       "  'timestamp': '2024-07-10T11:11:07.1107245Z',\n",
       "  'channelId': 'directline',\n",
       "  'from': {'id': 'BotId-zf4fwhz3gdn64', 'name': 'BotId-zf4fwhz3gdn64'},\n",
       "  'conversation': {'id': '9wah6oavbyqHdVAOR5hBEx-au'},\n",
       "  'text': 'Tool: docsearch',\n",
       "  'inputHint': 'acceptingInput',\n",
       "  'replyToId': '9wah6oavbyqHdVAOR5hBEx-au|0000001'},\n",
       " {'type': 'message',\n",
       "  'id': '9wah6oavbyqHdVAOR5hBEx-au|0000003',\n",
       "  'timestamp': '2024-07-10T11:11:08.0064298Z',\n",
       "  'channelId': 'directline',\n",
       "  'from': {'id': 'BotId-zf4fwhz3gdn64', 'name': 'BotId-zf4fwhz3gdn64'},\n",
       "  'conversation': {'id': '9wah6oavbyqHdVAOR5hBEx-au'},\n",
       "  'text': \"☑\\nInvoking: `docsearch` with `{'query': 'CLP'}`\\n\\n\\n ...\",\n",
       "  'inputHint': 'acceptingInput',\n",
       "  'replyToId': '9wah6oavbyqHdVAOR5hBEx-au|0000001'},\n",
       " {'type': 'message',\n",
       "  'id': '9wah6oavbyqHdVAOR5hBEx-au|0000004',\n",
       "  'timestamp': '2024-07-10T11:11:09.3402124Z',\n",
       "  'channelId': 'directline',\n",
       "  'from': {'id': 'BotId-zf4fwhz3gdn64', 'name': 'BotId-zf4fwhz3gdn64'},\n",
       "  'conversation': {'id': '9wah6oavbyqHdVAOR5hBEx-au'},\n",
       "  'text': \"I'm sorry, I cannot provide information on that topic.\",\n",
       "  'inputHint': 'acceptingInput',\n",
       "  'replyToId': '9wah6oavbyqHdVAOR5hBEx-au|0000001'}]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "activities"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b677916-67f8-4259-b086-66e5f3d2c357",
   "metadata": {},
   "source": [
    "## Create helper functions to talk to the API asyncronously"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38dc8571-5b31-49ee-b953-88aaa3a63561",
   "metadata": {},
   "source": [
    "These functions below define a simple system for asynchronously sending a message to a bot, waiting for a response, and then continuously checking for and printing new messages from the bot for a specified period. It uses aiohttp for asynchronous HTTP requests, allowing it to non-blockingly wait for responses from the bot and enforce a timeout if no new messages are received within the expected timeframe.\n",
    "\n",
    "Modify these at your will"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "78761c96-fded-4bbd-a11f-94f92d71a597",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to send a message to the bot service API.\n",
    "async def send_message(base_url, conversation_id, headers, question):\n",
    "    # Construct the URL for sending a message to the bot.\n",
    "    send_message_url = f\"{base_url}/conversations/{conversation_id}/activities\"\n",
    "    \n",
    "    # Prepare the timestamp, timezone, and locale for the message.\n",
    "    local_timestamp = datetime.datetime.now(pytz.timezone('America/New_York'))\n",
    "    local_timezone = str(local_timestamp.tzinfo)\n",
    "    locale = \"en-US\"\n",
    "\n",
    "    # Define the message payload, including the question and additional data.\n",
    "    message = {\n",
    "        \"type\": \"message\",\n",
    "        \"from\": {\"id\": \"user\"},\n",
    "        \"text\": question,\n",
    "        \"channelData\": {\n",
    "            \"local_timestamp\": local_timestamp.strftime(\"%I:%M:%S %p, %A, %B %d of %Y\"),\n",
    "            \"local_timezone\": local_timezone,\n",
    "            \"locale\": locale\n",
    "        },\n",
    "        # Example structure for sending an attachment, commented out here.\n",
    "        # \"attachments\": [\n",
    "        #     {\n",
    "        #         \"contentType\": \"image/jpeg\",\n",
    "        #         \"contentUrl\": \"https://example.com/image.jpg\",\n",
    "        #         \"name\": \"image.jpg\"\n",
    "        #     }\n",
    "        # ]\n",
    "    }\n",
    "    \n",
    "    # Use an asynchronous HTTP session to send the message.\n",
    "    async with aiohttp.ClientSession() as session:\n",
    "        async with session.post(send_message_url, headers=headers, json=message) as response:\n",
    "            print(\"Message sent status code:\", response.status)\n",
    "            response_text = await response.text()\n",
    "            print(\"Response text:\", response_text)\n",
    "\n",
    "            \n",
    "# Function to filter and print the last bot responses to the most recent user message.\n",
    "async def print_last_bot_responses(activities, bot_id):\n",
    "    last_user_msg_index = None\n",
    "    # Iterate through activities in reverse to find the last user message.\n",
    "    for index, msg in enumerate(reversed(activities)):\n",
    "        if msg['from']['id'] != bot_id:\n",
    "            last_user_msg_index = len(activities) - 1 - index\n",
    "            break\n",
    "\n",
    "    messages_to_print = []\n",
    "    # If a user message was found, collect all subsequent bot messages.\n",
    "    if last_user_msg_index is not None:\n",
    "        for msg in activities[last_user_msg_index + 1:]:\n",
    "            if msg['from']['id'] == bot_id:\n",
    "                messages_to_print.append(msg['text'])\n",
    "    \n",
    "    return messages_to_print\n",
    "\n",
    "\n",
    "# Main function to send a question to the bot and print responses.\n",
    "async def send_question_and_check_activities(base_url, bot_id, conversation_id, headers, question, timeout=30):\n",
    "    # Send the initial question to the bot.\n",
    "    await send_message(base_url, conversation_id, headers, question)\n",
    "    \n",
    "    async with aiohttp.ClientSession() as session:\n",
    "        last_printed_activity_id = None\n",
    "        # Record the time when the last message was received to enforce the timeout.\n",
    "        last_message_time = time.time()\n",
    "\n",
    "        while True:\n",
    "            current_time = time.time()\n",
    "            # Check if the specified timeout has elapsed without new messages.\n",
    "            if current_time - last_message_time > timeout:\n",
    "                print(f\"{timeout} seconds have elapsed without new messages. Exiting...\")\n",
    "                break\n",
    "\n",
    "            # Construct the URL to get conversation activities.\n",
    "            get_activities_url = f\"{base_url}/conversations/{conversation_id}/activities\"\n",
    "            \n",
    "            # Use an asynchronous HTTP session to fetch activities.\n",
    "            async with session.get(get_activities_url, headers=headers) as response:\n",
    "                activities = await response.json()\n",
    "                activities = activities[\"activities\"]\n",
    "                new_messages = await print_last_bot_responses(activities, bot_id)\n",
    "                \n",
    "                # Check for new messages from the bot since the last printed message.\n",
    "                if new_messages:\n",
    "                    last_activity_id = activities[-1]['id']\n",
    "                    if last_activity_id != last_printed_activity_id:\n",
    "                        for message in new_messages:\n",
    "                            print(message)\n",
    "                        # Update tracking variables with the latest message details.\n",
    "                        last_printed_activity_id = last_activity_id\n",
    "                        last_message_time = current_time\n",
    "\n",
    "            # Wait for a short period before checking for new messages again.\n",
    "            await asyncio.sleep(1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5b77069-24ce-417e-abac-eff6ee52ab5a",
   "metadata": {},
   "source": [
    "## Talk to the bot API "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c50b5ae7-65df-4c92-9f10-edf5536a4b42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Converstion id: II0A5i5aSjXHN8VsJuildi-au\n"
     ]
    }
   ],
   "source": [
    "# Start a conversation\n",
    "start_conversation_url = f\"{base_url}/conversations\"\n",
    "headers = {\"Authorization\": f\"Bearer {direct_line_secret}\"}\n",
    "\n",
    "response = requests.post(start_conversation_url, headers=headers)\n",
    "conversation_id = response.json()[\"conversationId\"]\n",
    "print('Converstion id:', conversation_id)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf686517-f0a7-4401-a2e8-0c7c9452aa6f",
   "metadata": {},
   "source": [
    "### Ask the first question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8fd36878-03e5-468f-b4cc-d04a720b5bff",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"sqlsearch, what is the country with the most deaths in 2020?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3367361f-3372-47a5-a156-45fb7d6cfd8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Message sent status code: 200\n",
      "Response text: {\n",
      "  \"id\": \"II0A5i5aSjXHN8VsJuildi-au|0000001\"\n",
      "}\n",
      "Tool: sqlsearch\n",
      "☑\n",
      "Invoking: `sql_db_list_tables` with `{}`\n",
      "\n",
      "\n",
      " ...\n",
      "☑\n",
      "Invoking: `sql_db_schema` with `{'table_names': 'covidtracking'}`\n",
      "\n",
      "\n",
      " ...\n",
      "☑\n",
      "Invoking: `sql_db_query` with `{'query': \"SELECT state, SUM(death) AS total_deaths FROM covidtracking WHERE date LIKE '2020%' GROUP BY state ORDER BY total_deaths DESC\"}`\n",
      "\n",
      "\n",
      " ...\n",
      "The state with the most deaths in 2020 was New York (NY), with a total of 6,530,995 deaths.\n",
      "\n",
      "This information was obtained by querying the `covidtracking` table for the `state` and the sum of `death` where the date starts with '2020'. The results were then grouped by state and ordered by the total number of deaths in descending order. The SQL query used for this purpose is as follows:\n",
      "\n",
      "```sql\n",
      "SELECT state, SUM(death) AS total_deaths\n",
      "FROM covidtracking\n",
      "WHERE date LIKE '2020%'\n",
      "GROUP BY state\n",
      "ORDER BY total_deaths DESC\n",
      "```\n",
      "\n",
      "This query returned the total number of deaths for each state in 2020, and New York (NY) had the highest number of deaths.\n",
      "\n",
      "References:\n",
      "- Data source: [covidtracking table](https://example.com/covidtracking)\n",
      "25 seconds have elapsed without new messages. Exiting...\n"
     ]
    }
   ],
   "source": [
    "await send_question_and_check_activities(base_url, bot_id, conversation_id, headers, QUESTION, timeout=timeout)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b63eb0ea-40c1-4efd-aa66-ad7599d19400",
   "metadata": {},
   "source": [
    "### Now a follow up question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "17dc446f-0bff-4704-93a3-857c4b294930",
   "metadata": {},
   "outputs": [],
   "source": [
    "FOLLOWUP_QUESTION = \"interesting, and about the state with the least?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e6143c06-0cf0-4f0a-9077-36a5604d40c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Message sent status code: 200\n",
      "Response text: {\n",
      "  \"id\": \"II0A5i5aSjXHN8VsJuildi-au|0000007\"\n",
      "}\n",
      "Tool: sqlsearch\n",
      "☑\n",
      "Invoking: `sql_db_list_tables` with `{}`\n",
      "\n",
      "\n",
      " ...\n",
      "☑\n",
      "Invoking: `sql_db_schema` with `{'table_names': 'covidtracking'}`\n",
      "\n",
      "\n",
      " ...\n",
      "☑\n",
      "Invoking: `sql_db_query` with `{'query': \"SELECT state, SUM(death) AS total_deaths FROM covidtracking WHERE date LIKE '2020%' GROUP BY state ORDER BY total_deaths ASC\"}`\n",
      "\n",
      "\n",
      " ...\n",
      "The state with the lowest total deaths from COVID-19 in 2020 was American Samoa (AS) with 0 deaths.\n",
      "\n",
      "This information was obtained by querying the `covidtracking` table for the `state` and the sum of `death` where the date starts with '2020'. The results were then grouped by state and ordered by the total number of deaths in ascending order. The SQL query used for this purpose is as follows:\n",
      "\n",
      "```sql\n",
      "SELECT state, SUM(death) AS total_deaths\n",
      "FROM covidtracking\n",
      "WHERE date LIKE '2020%'\n",
      "GROUP BY state\n",
      "ORDER BY total_deaths ASC\n",
      "LIMIT 1\n",
      "```\n",
      "\n",
      "This query returned the state with the lowest number of deaths in 2020, which was American Samoa (AS) with 0 deaths.\n",
      "\n",
      "References:\n",
      "- Data source: [covidtracking table](https://example.com/covidtracking)\n",
      "25 seconds have elapsed without new messages. Exiting...\n"
     ]
    }
   ],
   "source": [
    "await send_question_and_check_activities(base_url, bot_id, conversation_id, headers, FOLLOWUP_QUESTION, timeout=timeout)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a37dc1c4-fbc1-4da5-9cba-042ffba39c40",
   "metadata": {},
   "source": [
    "# Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67ba812b-e2c3-40c3-a80d-7ad8befccf02",
   "metadata": {},
   "source": [
    "In this notebook, we've explored how to programmatically communicate with the Bot Service API.\n",
    "\n",
    "An important aspect to note regarding the responses from the API is that it consistently returns the entire conversation history, not just the latest message. This behavior is evident in the following code snippet from the helper functions mentioned earlier:\n",
    "\n",
    "```python\n",
    "async with session.get(get_activities_url, headers=headers) as response:\n",
    "                activities = await response.json()\n",
    "                activities = activities[\"activities\"]\n",
    "                new_messages = await print_last_bot_responses(activities, bot_id)\n",
    "```\n",
    "Go ahead and print out the variable `activities`; you'll observe this for yourself.\n",
    "\n",
    "This characteristic opens up the possibility of developing a simpler memory system. By including previous messages along with the current question, it's feasible to manage conversational context using the frontend without the need for persistent storage solutions like Cosmos DB in the backend."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db72ef97-fae5-47ee-bfaf-5ca2feaf81bd",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "\n",
    "In the next notebook, we will venture into creating a different type of Backend API, this time utilizing FastAPI and LangServe. This approach will also enable us to incorporate streaming capabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31ead94f-ee43-4098-9d4a-930f955420bf",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}