Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove training with iterable datasets? #49

Open
VarIr opened this issue Feb 9, 2021 · 1 comment
Open

Remove training with iterable datasets? #49

VarIr opened this issue Feb 9, 2021 · 1 comment

Comments

@VarIr
Copy link
Collaborator

VarIr commented Feb 9, 2021

As discussed in #43 training with iterable datasets is rarely used. In particular, training without shuffling might never be useful.
Due to that, the corresponding functionality is not subject to many tests, and was broken for some time, until this was pointed out in PR #43.

While for the time being, the bug is fixed, and additional tests have been put in place, we might want to remove these functions altogether. This could reduce maintenance cost, and improve quality.

@VarIr
Copy link
Collaborator Author

VarIr commented Mar 5, 2021

While at it, we might want to let our ShuffledProteinIterableDataset inherit from the BufferedShuffleDataset newly introduced in PyTorch 1.8. From a first glance, it does exactly the same thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant