Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bootleg takes way too much space and memory #156

Open
gcampax opened this issue Jun 16, 2021 · 3 comments
Open

Bootleg takes way too much space and memory #156

gcampax opened this issue Jun 16, 2021 · 3 comments
Assignees
@Mehrad0711
Labels
bootleg Issues with NER and Bootleg bug Something isn't working P2 We need to fix it (backlog) performance Performance issues: resource usage, scaling, GPU efficiency server Issues with serving and dynamic inference-time

Comments

@gcampax
Copy link
Contributor

gcampax commented Jun 16, 2021

I had to bump up the ephemeral storage of BART models with Bootleg to 55G. It takes ~20 minutes for a model to start for inference, as it downloads and then loads in memory gigs and gigs of "stuff". This is simply not feasible.
@gcampax gcampax added the bug Something isn't working label Jun 16, 2021
@Mehrad0711 Mehrad0711 mentioned this issue Jun 29, 2021
Merged
@gcampax gcampax added server Issues with serving and dynamic inference-time bootleg Issues with NER and Bootleg performance Performance issues: resource usage, scaling, GPU efficiency labels Jul 15, 2021
@Mehrad0711
Copy link
Member

Addressed in #166

@gcampax
Copy link
Contributor Author

gcampax commented Aug 4, 2021

No it wasn't! You only addressed the memory part (and only in the sense that memory is reclaimable, not in the sense that memory is not actually used), startup time and disk usage are still to be fixed.

@gcampax gcampax reopened this Aug 4, 2021
@Mehrad0711
Copy link
Member

I'm afraid we need to add more assignees then! Addressing disk space requires more than surface changes in bootleg code.

@nrser nrser added the P2 We need to fix it (backlog) label Aug 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Mehrad0711 Mehrad0711

Labels
bootleg Issues with NER and Bootleg bug Something isn't working P2 We need to fix it (backlog) performance Performance issues: resource usage, scaling, GPU efficiency server Issues with serving and dynamic inference-time
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nrser @gcampax @Mehrad0711