Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/237 #241

Open
wants to merge 53 commits into
base: dev
Choose a base branch
from
Open

Issue/237 #241

wants to merge 53 commits into from

Commits on Sep 11, 2024

  1. DEV: Created standalone class for Random Forest Models (#237)

    FEAT: Implemented RF class method for fitting the model
    
    FEAT: Implemented RF class method for obtaining importance analysis
    from a fitted RF
    
    FEAT: Implemented RF class method for returning oob error
    
    FEAT: Implemented RF class method for obtaining FDR
    from a fitted model
    
    FEAT: Implemented RF class method for exporting forest to JSON
    
    REFACTOR: Make RF model available at package level
    
    CHORE: Added type checking to all methods
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    5edfbfa View commit details
    Browse the repository at this point in the history
  2. DEV: Updated varspark python wrapper (#237)

    REFACTOR: Removed FeatureSource and
    ImportanceAnalysis classes from core
    
    REFACTOR: Added FeatureSource import so features
    can be returned as a class instantiation
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    80a9c59 View commit details
    Browse the repository at this point in the history
  3. DEV: Created standalone FeatureSource class in separate file (#237)

    REFACTOR: Removed imp analysis and model training
    
    FEAT: Added conversion from feature to RDD (python)
    
    FEAT: Added conversion from feature to RDD (scala)
    
    CHORE: Added type checking
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    23520ec View commit details
    Browse the repository at this point in the history
  4. REFACTOR: Remove unecessary hail import for hail rf wrapper

    due to import order warning (#237)
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4560998 View commit details
    Browse the repository at this point in the history
  5. DEV: Created standalone ImportanceAnalysis class in

    separate wrapper file (#237)
    
    REFACTOR: Updated important_variables and variable_importance
    methods to convert to pandas DataFrames
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b8b39fd View commit details
    Browse the repository at this point in the history
  6. DEV: Updated ImportanceAnalysis scala class (#237)

    REFACTOR: Removed model training from object instantation and
    updated class to accept a model as a parameter
    
    REFACTOR: Added normalisation as an optional parameter for
    variable importance methods
    
    FEAT: Updated variableImportance method to include splitCount in return as it is required for local FDR analysis
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4bfaac9 View commit details
    Browse the repository at this point in the history
  7. DEV: Created scala function that trains a forest

    and passes back to python context (#237)
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    0fc736f View commit details
    Browse the repository at this point in the history
  8. REFACTOR: Removed model definition and training

    from importAnalysis method of AnalyticsFunctions (#237)
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    ea069d6 View commit details
    Browse the repository at this point in the history
  9. DEV: Create no-hail equivalent of JSON model export (#237)

    FIX: Update export function to process trees in batches,
    instead of collecting the whole forest as a map as this
    led to OOM errors on large forests
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    e08f12a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    ddc5912 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    f6d40d4 View commit details
    Browse the repository at this point in the history
  12. DEV: Update python unit testing (#237)

    REFACTOR: Refactor to mirror changes to python wrapper
    
    FEAT: Include FDR calculation in unit test
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    3356d9a View commit details
    Browse the repository at this point in the history
  13. DEV: Create no hail lfdr class (#237)

    FEAT: Implement function for manhattan plotting negative log p values
    NickEdwards7502 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    59f40bc View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    3f8066b View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    de29b45 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. DEV: Add covariate import wrapper function (#237)

    STYLE: Format with black
    NickEdwards7502 committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    fe2db4c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a9b9570 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3ea4c8c View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Configuration menu
    Copy the full SHA
    8f11e62 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d671f35 View commit details
    Browse the repository at this point in the history
  3. DEV: Add wrapper functions for covariate support (#237)

    FEAT: Add wrapper class for importing covariates
    
    FEAT: Add wrapper class for unioning features and covariates
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    209a463 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b94afcc View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    04daae2 View commit details
    Browse the repository at this point in the history
  6. DEV: Update lfdr to support covariates (#237)

    REFACTOR: Include covariate filtering in manhattan plot function
    
    STYLE: Format with black (#237)
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    30732ba View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3381e68 View commit details
    Browse the repository at this point in the history
  8. DEV: Update VSContext to support covariates (#237)

    FEAT: Add functions for importing std and transposed CSVs
    
    FEAT: Add function for unioning features and covariates
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    37f4193 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    dfae3c2 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    9733844 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    dd32e0f View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    769ce76 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    e9a23cf View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    4379b0a View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    d2048d0 View commit details
    Browse the repository at this point in the history
  16. REFACTOR: Update pairwise operation tests based on import changes (#237)

    Reference changed to importTransposedCSV
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    2416b8e View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    1529bd8 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    12d6137 View commit details
    Browse the repository at this point in the history
  19. DEV: Replace to_df functionality with head (#237)

    REFACTOR: Remove python component of converting Feature RDD to pandas
    
    FEAT: Add RDD slice to DF function
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    4df6e32 View commit details
    Browse the repository at this point in the history
  20. DEV: Update FeatureSource dataframe conversion (#237)

    REFACTOR: Remove conversion of whole RDD to DataFrame
    
    FEAT: Add function for slicing rows and columns and converting to DF
    NickEdwards7502 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    b1fe760 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    4506139 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Configuration menu
    Copy the full SHA
    07cd144 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Configuration menu
    Copy the full SHA
    cef13f4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b6f4e3b View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Configuration menu
    Copy the full SHA
    9b7f83b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f66f445 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e6e637c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8f82d29 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1be8d66 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    bd53f27 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    45ae200 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    fe70285 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Configuration menu
    Copy the full SHA
    279bd5b View commit details
    Browse the repository at this point in the history
  2. DEV: Integrate bgzipped file support in VCF import API (#237)

    * .bgz loader function implemented by Christina
    NickEdwards7502 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5ad8cc0 View commit details
    Browse the repository at this point in the history
  3. DEV: Implement imputation for VCF features (#237)

    * Update python wrapper to include imputation strategy parameter
    
    * Update scala API to pass imputation strategy to VCFFeatureSource
    
    * Create functions to handle mode and zero imputation strategies
    
    * Added imputation strategy to test cases
    
    * Added imputation strategy to FeatureSource cli
    
    * Remove sparkPar from test cases due to changes in class signature
    
    * Updated DefVariantToFeatureConverterTest to use zeros imputation
    NickEdwards7502 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    b686d75 View commit details
    Browse the repository at this point in the history