Skip to content

Latest commit

 

History

History
99 lines (81 loc) · 6.57 KB

readme.md

File metadata and controls

99 lines (81 loc) · 6.57 KB
Raw

Covid Severity Forecasting

Data and models (updated daily) for forecasting COVID-19 severity for individual counties and hospitals in the US. The data includes confirmed cases/deaths, demographics, risk factors, social distancing data, and much more.

Table of contents
Overview • Quickstart • Acknowledgements
Resources
Data csv • Paper • Website • Modeling docs • Dashboard code

Overview

Note: This repo is actively maintained - for any questions, please file an issue.

  • Data (updated daily): We have cleaned, merged, and documented a large corpus of hospital- and county-level data from a variety of public sources to aid data science efforts to combat COVID-19.
    • At the hospital level, the data include the location of the hospital, the number of ICU beds, the total number of employees, the hospital type, and contact information
    • At the county level, our data include socioeconomic factors, social distancing scores, and COVID-19 cases/deaths from USA Facts and NYT
    • Easily downloadable as processed csv or full pipeline
    • Extensive documentation available here
  • Paper link: "Curating a COVID-19 data repository and forecasting county-level death counts in the United States"
  • Project website: http://covidseverity.com/
  • Modeling: Using this data, we have developed a short-term (3-5 days) forecasting model for mortality at the county level. This model combines a county-specific exponential growth model and a shared exponential growth model through a weighted average, where the weights depend on past prediction accuracy.
  • Severity index: The Covid pandemic severity index (CPSI) is designed to help aid the distribution of medical resources to hospitals. It takes on three values (3: High, 2: Medium, 1: Low), indicating the severity of the covid-19 outbreak for a hospital on a certain day. It is calculated in three steps.
    1. county-level predictions for number of deaths are modeled
    2. county-level predictions are allocated to hospitals within counties proportional the their total number of employees
    3. final value is decided by thresholding the number of cumulative predicted deaths for a hospital (=current recorded deaths + predicted future deaths)

Quickstart with the data + models

Can download, load, and merge the data via:

import load_data
# first time it runs, downloads and caches the data
df = load_data.load_county_level(data_dir='/path/to/data')
  • for more data details, see ./data/readme.md
  • see also the quickstart notebook
  • we are constantly monitoring and adding new data sources (+ relevant data news here)
  • output from running the daily updates is stored here

To get deaths predictions for our current best-performing model, the simplest way is to call the add_preds function (for more details, see ./modeling/readme.md)

from modeling.fit_and_predict import add_preds
df = add_preds(df, NUM_DAYS_LIST=[1, 3, 5]) # adds keys like "Predicted Deaths 1-day", "Predicted Deaths 3-day"
# NUM_DAYS_LIST is list of number of days in the future to predict

Related county-level projects

Acknowledgements

The UC Berkeley Departments of Statistics, EECS led by Professor Bin Yu (group members are all alphabetical by last name)

  • Yu group team (Data/modeling): Nick Altieri, Rebecca Barter, James Duncan, Raaz Dwivedi, Karl Kumbier, Xiao Li, Robbie Netzorg, Briton Park, Chandan Singh (student lead), Yan Shuo Tan, Tiffany Tang, Yu Wang
    • Summer team: Abhineet Agarwal, Maya Shen, Danqing Wang, Chao Zhang
  • Response4Life (Organization/distribution) team and volunteers, particularly Don Landwirth and Rick Brennan
  • Medical team (Advice from a medical perspective): Roger Chaufournier, Aaron Kornblith, David Jaffe
  • Hospital-info collection: Matthew Shen, Anthony Rio, Miles Bishop, Josh Davis, and Dylan Goetting
  • Kolak group team (Geospatial visualization): Qinyun Lin
  • Support from Google: Cat Allman and Peter Norvig
  • Shen Group team (IEOR): Junyu Cao, Shunan Jiang, Pelagie Elimbi Moudio
  • Helpful input from many including: SriSatish Ambati, Rob Crockett, Tina Elassia-Rad, Marty Elisco, Nick Jewell, Valerie Isham, Valerie Karplus, Andreas Lange, Ying Lu, Samuel Scarpino, Jas Sekhon, Phillip Stark, Jacob Steinhardt, Suzanne Tamang, Brian Yandell, Tarek Zohdi
  • Thanks to support from AWS and Google
  • Additionally, we would like to thank our sources, which can be found in the data readme

To reference, please cite the paper:

@article{altieri2020Curating,
  journal = {Harvard Data Science Review},
  doi = {10.1162/99608f92.1d4e0dae},
  note = {https://hdsr.mitpress.mit.edu/pub/p6isyf0g},
  title = {Curating a COVID-19 Data Repository and Forecasting County-Level DeathCounts in the United States},
  url = {https://hdsr.mitpress.mit.edu/pub/p6isyf0g},
  author = {Altieri, Nick and Barter, Rebecca L and Duncan, James and Dwivedi, Raaz and Kumbier, Karl and Li, Xiao and Netzorg, Robert and Park, Briton and Singh, Chandan and Tan, Yan Shuo and Tang, Tiffany and Wang, Yu and Zhang, Chao and Yu, Bin},
  date = {2020-11-03},
  year = {2020},
  month = {11},
  day = {3},
}