article_classifier_test_1.3.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running integration test with query: 'Assisted+dying', max articles: 5, model: llama3\n",
      "Total articles found: 3878\n",
      "Fetching up to 5 articles...\n",
      "Successfully fetched 5 articles.\n",
      "Articles before processing:\n",
      "ID: 39167528, Title: The Impact of Legalizing Medical Aid in Dying on P...\n",
      "ID: 39160544, Title: Non-invasive technology to assess hydration status...\n",
      "ID: 39157533, Title: Readiness of nurses when faced with a patient's de...\n",
      "ID: 39157418, Title: 'There is no such word as palliative care for us a...\n",
      "ID: 39152645, Title: The double awareness of the wish to hasten death a...\n",
      "\n",
      "Processing article 1 of 5...\n",
      "Article ID: 39167528\n",
      "DOI: 10.1089/jpm.2023.0706\n",
      "Title: The Impact of Legalizing Medical Aid in Dying on Patient Trust: A Randomized Controlled Survey Study.\n",
      "Error processing article 39167528: 'NoneType' object is not subscriptable\n",
      "--------------------------------------------------\n",
      "\n",
      "Processing article 2 of 5...\n",
      "Article ID: 39160544\n",
      "DOI: 10.1186/s12904-024-01542-z\n",
      "Title: Non-invasive technology to assess hydration status in advanced cancer to explore relationships between fluid status and symptoms: an observational study using bioelectrical impedance analysis.\n",
      "Abstract: Oral fluid intake decreases in advanced cancer in the dying phase of illness. There is inadequate ev...\n",
      "Study Type: ** Empirical\n",
      "Study Type Justification: ** The study is empirical because it involves collecting data through observational methods and statistical analysis to explore relationships between fluid status and symptoms in advanced cancer patients. The study aims to provide new insights into the assessment and management of hydration in this population, which is typical of empirical research.\n",
      "Research Methods: ** Bioelectrical impedance analysis (BIA), observational study\n",
      "Research Methods Justification: ** The use of BIA as a body composition assessment tool allows researchers to non-invasively measure changes in fluid status in advanced cancer patients. An observational study design enables the collection of data on clinical variables and hydration status, enabling the exploration of relationships between these factors.\n",
      "--------------------------------------------------\n",
      "\n",
      "Processing article 3 of 5...\n",
      "Article ID: 39157533\n",
      "DOI: 10.3389/fpubh.2024.1399025\n",
      "Title: Readiness of nurses when faced with a patient's death.\n",
      "Abstract: The death of a patient negatively affects the professional dimension of nurses' functioning and also...\n",
      "Study Type: ** Empirical\n",
      "Study Type Justification: ** The study appears to be empirical because it investigates the experiences and emotions of nurses when faced with a patient's death, which suggests a quantitative or qualitative analysis of real-life data. The title \"Readiness of nurses when faced with a patient's death\" also implies an investigation into the phenomenon rather than a theoretical discussion.\n",
      "Research Methods: ** Surveys, interviews\n",
      "Research Methods Justification: ** The study likely employed surveys and interviews to gather data on the experiences and emotions of nurses after a patient's death. This combination of methods allows for both quantitative and qualitative data collection, which is useful in exploring the complex phenomenon of nurses' responses to patient death.\n",
      "--------------------------------------------------\n",
      "\n",
      "Processing article 4 of 5...\n",
      "Article ID: 39157418\n",
      "DOI: 10.1177/26323524241272102\n",
      "Title: 'There is no such word as palliative care for us at the moment': A mixed-method study exploring the perceptions of healthcare professionals on the need for palliative care in Bhutan.\n",
      "Abstract: The need for palliative care is ever-increasing globally. However, it is least developed or not avai...\n",
      "Study Type: ** Empirical\n",
      "Study Type Justification: ** The study is empirical because it involves data collection and analysis from healthcare professionals in Bhutan, rather than relying solely on theoretical frameworks or literature reviews. The study's focus on exploring perceptions and experiences also suggests an empirical approach.\n",
      "Research Methods: ** Mixed-methods survey and interviews, with a total of 50 participants (34 surveys and 16 interviews)\n",
      "Research Methods Justification: ** The use of mixed-methods research allows the authors to gather both quantitative and qualitative data, providing a more comprehensive understanding of healthcare professionals' perceptions on palliative care in Bhutan. The combination of survey and interview methods also enables the collection of both general attitudes and specific experiences, which can provide richer insights into the topic.\n",
      "--------------------------------------------------\n",
      "\n",
      "Processing article 5 of 5...\n",
      "Article ID: 39152645\n",
      "DOI: 10.1177/02692163241269689\n",
      "Title: The double awareness of the wish to hasten death and the will to live: A secondary analysis of outlier patients from a mixed-methods study.\n",
      "Abstract: Patients with serious illness frequently report (temporary) wishes to hasten death. Even until the e...\n",
      "Study Type: ** Empirical\n",
      "Study Type Justification: ** The study is empirical because it analyzes existing data from a mixed-methods study, rather than proposing or testing theoretical frameworks. The authors present findings based on their examination of outlier patients from an original study, suggesting that they are working with real-world data to explore the relationship between the wish to hasten death and the will to live.\n",
      "Research Methods: ** Mixed-methods study, secondary analysis of outlier patients\n",
      "Research Methods Justification: ** The use of a mixed-methods approach allows the authors to combine both quantitative and qualitative data from the original study. By analyzing outlier patients, they can gain a deeper understanding of complex phenomena that may not be captured by typical statistical analyses. This methodological choice enables the researchers to explore the relationship between the wish to hasten death and the will to live in a nuanced way, which is particularly important given the abstract's focus on the coexistence of these seemingly opposing desires.\n",
      "--------------------------------------------------\n",
      "\n",
      "Integration test complete! Processed 5 articles.\n",
      "Results saved to pubmed_ollama_test_results_20240821_194305.json\n",
      "Results saved to pubmed_ollama_test_results_20240821_194305.csv\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import xml.etree.ElementTree as ET\n",
    "import time\n",
    "import json\n",
    "import csv\n",
    "from datetime import datetime\n",
    "import re\n",
    "from tenacity import retry, stop_after_attempt, wait_fixed\n",
    "\n",
    "def fetch_pubmed_articles(query, max_results=5):\n",
    "    base_url = \"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/\"\n",
    "    \n",
    "    search_url = f\"{base_url}esearch.fcgi?db=pubmed&term={query}&usehistory=y&retmode=json\"\n",
    "    search_response = requests.get(search_url).json()\n",
    "    \n",
    "    total_count = int(search_response['esearchresult']['count'])\n",
    "    webenv = search_response['esearchresult']['webenv']\n",
    "    query_key = search_response['esearchresult']['querykey']\n",
    "    \n",
    "    print(f\"Total articles found: {total_count}\")\n",
    "    print(f\"Fetching up to {max_results} articles...\")\n",
    "\n",
    "    fetch_url = f\"{base_url}efetch.fcgi?db=pubmed&query_key={query_key}&WebEnv={webenv}&retmode=xml&retmax={max_results}\"\n",
    "    fetch_response = requests.get(fetch_url)\n",
    "    \n",
    "    root = ET.fromstring(fetch_response.content)\n",
    "    \n",
    "    articles = []\n",
    "    for article in root.findall(\".//PubmedArticle\"):\n",
    "        pmid = article.find(\".//PMID\").text if article.find(\".//PMID\") is not None else \"ID not available\"\n",
    "        title_element = article.find(\".//ArticleTitle\")\n",
    "        title = title_element.text if title_element is not None else \"Title not available\"\n",
    "        abstract_element = article.find(\".//Abstract/AbstractText\")\n",
    "        abstract = abstract_element.text if abstract_element is not None else \"Abstract not available\"\n",
    "        doi_element = article.find(\".//ArticleId[@IdType='doi']\")\n",
    "        doi = doi_element.text if doi_element is not None else \"DOI not available\"\n",
    "        \n",
    "        articles.append({\n",
    "            'id': pmid,\n",
    "            'doi': doi,\n",
    "            'title': title,\n",
    "            'abstract': abstract\n",
    "        })\n",
    "    \n",
    "    return articles\n",
    "\n",
    "def preprocess_article(article):\n",
    "    # Check if the article is None\n",
    "    if article is None:\n",
    "        return None\n",
    "\n",
    "    # Check if 'abstract' key exists and is not None\n",
    "    if 'abstract' not in article or article['abstract'] is None:\n",
    "        return None\n",
    "\n",
    "    # Check if the abstract is too short or not available\n",
    "    if len(article['abstract']) < 50 or article['abstract'] == \"Abstract not available.\":\n",
    "        return None\n",
    "    \n",
    "    # Check if 'title' key exists and is not None\n",
    "    if 'title' not in article or article['title'] is None:\n",
    "        return None\n",
    "\n",
    "    # Check if the title is missing\n",
    "    if article['title'] == \"Title not available.\":\n",
    "        return None\n",
    "    \n",
    "    return article\n",
    "\n",
    "@retry(stop=stop_after_attempt(3), wait=wait_fixed(2))\n",
    "def process_article_with_ollama(article, model_name=\"llama3\"):\n",
    "    url = \"http://localhost:11434/api/generate\"\n",
    "    \n",
    "    prompt = f\"\"\"\n",
    "    Analyze the following research article on assisted dying:\n",
    "\n",
    "    Title: {article['title']}\n",
    "\n",
    "    Abstract: {article['abstract']}\n",
    "\n",
    "    Please provide a detailed analysis addressing the following points. Format your response exactly as shown below:\n",
    "\n",
    "    Study Type: [Empirical/Theoretical]\n",
    "\n",
    "    Study Type Justification: [Your explanation here]\n",
    "\n",
    "    Research Methods: [List the specific methods used, separated by commas]\n",
    "\n",
    "    Research Methods Justification: [Your explanation here]\n",
    "\n",
    "    Ensure your response follows this exact format for easy parsing.\n",
    "\n",
    "    Response:\n",
    "    \"\"\"\n",
    "    \n",
    "    payload = {\n",
    "        \"model\": model_name,\n",
    "        \"prompt\": prompt,\n",
    "        \"stream\": False\n",
    "    }\n",
    "    \n",
    "    try:\n",
    "        response = requests.post(url, json=payload)\n",
    "        response.raise_for_status()\n",
    "        result = response.json()['response']\n",
    "        parsed_result = parse_llm_response(result)\n",
    "        \n",
    "        # Add article metadata to the parsed result\n",
    "        parsed_result['article_id'] = article['id']\n",
    "        parsed_result['doi'] = article['doi']\n",
    "        parsed_result['title'] = article['title']\n",
    "        parsed_result['abstract'] = article['abstract']\n",
    "        \n",
    "        return parsed_result\n",
    "    except requests.exceptions.RequestException as e:\n",
    "        print(f\"Error processing article {article['id']}: {str(e)}\")\n",
    "        raise\n",
    "\n",
    "def parse_llm_response(response):\n",
    "    parsed_result = {\n",
    "        'study_type': \"Unable to determine\",\n",
    "        'study_type_justification': \"Unable to determine\",\n",
    "        'research_methods': \"Unable to determine\",\n",
    "        'research_methods_justification': \"Unable to determine\"\n",
    "    }\n",
    "    \n",
    "    # Use more flexible regex patterns\n",
    "    patterns = {\n",
    "        'study_type': r\"Study Type:\\s*(.+?)(?:\\n|$)\",\n",
    "        'study_type_justification': r\"Study Type Justification:\\s*(.+?)(?:\\n\\n|\\n[A-Z]|$)\",\n",
    "        'research_methods': r\"Research Methods:\\s*(.+?)(?:\\n|$)\",\n",
    "        'research_methods_justification': r\"Research Methods Justification:\\s*(.+?)(?:\\n\\n|$)\"\n",
    "    }\n",
    "    \n",
    "    for key, pattern in patterns.items():\n",
    "        match = re.search(pattern, response, re.DOTALL | re.IGNORECASE)\n",
    "        if match:\n",
    "            parsed_result[key] = match.group(1).strip()\n",
    "    \n",
    "    return parsed_result\n",
    "\n",
    "def validate_results(results):\n",
    "    validated_results = []\n",
    "    for result in results:\n",
    "        if all(value != \"Unable to determine\" for value in result.values()):\n",
    "            validated_results.append(result)\n",
    "        else:\n",
    "            print(f\"Incomplete result for article {result['article_id']}. Skipping.\")\n",
    "    return validated_results\n",
    "\n",
    "def save_results(results, base_filename):\n",
    "    # Save as JSON\n",
    "    json_filename = f\"{base_filename}.json\"\n",
    "    with open(json_filename, 'w') as f:\n",
    "        json.dump(results, f, indent=2)\n",
    "    print(f\"Results saved to {json_filename}\")\n",
    "\n",
    "    # Save as CSV\n",
    "    csv_filename = f\"{base_filename}.csv\"\n",
    "    \n",
    "    # Get all unique keys from all result dictionaries\n",
    "    fieldnames = set()\n",
    "    for result in results:\n",
    "        fieldnames.update(result.keys())\n",
    "    fieldnames = sorted(list(fieldnames))  # Sort field names for consistency\n",
    "\n",
    "    with open(csv_filename, 'w', newline='', encoding='utf-8') as f:\n",
    "        writer = csv.DictWriter(f, fieldnames=fieldnames)\n",
    "        writer.writeheader()\n",
    "        for result in results:\n",
    "            writer.writerow(result)\n",
    "    print(f\"Results saved to {csv_filename}\")\n",
    "\n",
    "def run_integration_test(query=\"Assisted+dying\", max_articles=5, model_name=\"llama3\"):\n",
    "    print(f\"Running integration test with query: '{query}', max articles: {max_articles}, model: {model_name}\")\n",
    "    \n",
    "    # Fetch articles\n",
    "    articles = fetch_pubmed_articles(query, max_articles)\n",
    "    print(f\"Successfully fetched {len(articles)} articles.\")\n",
    "    \n",
    "    # Add this line to print the articles before processing\n",
    "    print(\"Articles before processing:\")\n",
    "    for article in articles:\n",
    "        print(f\"ID: {article['id']}, Title: {article['title'][:50]}...\")\n",
    "\n",
    "    # Process articles with Ollama\n",
    "    results = []\n",
    "    for i, article in enumerate(articles, 1):\n",
    "        print(f\"\\nProcessing article {i} of {len(articles)}...\")\n",
    "        try:\n",
    "            result = process_article_with_ollama(article, model_name)\n",
    "            results.append(result)\n",
    "            print(f\"Article ID: {result['article_id']}\")\n",
    "            print(f\"DOI: {result['doi']}\")\n",
    "            print(f\"Title: {result['title']}\")\n",
    "            print(f\"Abstract: {result['abstract'][:100]}...\")  # Truncated for display\n",
    "            print(f\"Study Type: {result['study_type']}\")\n",
    "            print(f\"Study Type Justification: {result['study_type_justification']}\")\n",
    "            print(f\"Research Methods: {result['research_methods']}\")\n",
    "            print(f\"Research Methods Justification: {result['research_methods_justification']}\")\n",
    "        except Exception as e:\n",
    "            print(f\"Error processing article {article['id']}: {str(e)}\")\n",
    "        print(\"-\" * 50)\n",
    "        time.sleep(1)  # Be respectful to the Ollama API\n",
    "    \n",
    "    print(f\"\\nIntegration test complete! Processed {len(results)} articles.\")\n",
    "    return results\n",
    "\n",
    "# Run the integration test\n",
    "if __name__ == \"__main__\":\n",
    "    timestamp = datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n",
    "    base_filename = f\"pubmed_ollama_test_results_{timestamp}\"\n",
    "    \n",
    "    results = run_integration_test()\n",
    "    save_results(results, base_filename)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "myenv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}