Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Add resume logic option to pick up from individual pipeline steps #120

Open
Alfredo-Enrique opened this issue Jun 30, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@Alfredo-Enrique
Copy link
Member

Key feature request: To make the pipeline more robust and functional, we need to add additional logic that allows us to resume processing if individual pipelines that have not finished. As of right now, users might be tempted to relaunch the pipeline from the start if some of the individual downstream pipelines errored out (eg. call-mtSNV, call-sSV). As of now, users need to manually check outputs, and if something fails in a downstream pipeline, they have to make new individual inputs and group configs for all these samples.

A more harmonious/functional approach is to add logic to :

  1. Check if the individual pipeline steps are finished and copied the sample to /hot
  2. If No, relaunch this individual patient/sample group from the last pipeline not completed.

This will especially be useful in case of cluster instability, as we can just resume from later steps, and free up user time.

@Alfredo-Enrique Alfredo-Enrique added the enhancement New feature or request label Jun 30, 2023
@Alfredo-Enrique
Copy link
Member Author

@zhuchcn @yashpatel6 Let me know if I missed anything from our discussion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant