Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory usage ananse network #149

Open
JGASmits opened this issue Oct 27, 2021 · 0 comments
Open

memory usage ananse network #149

JGASmits opened this issue Oct 27, 2021 · 0 comments
Assignees

Comments

@JGASmits
Copy link
Contributor

When running ananse network with a single core (multi-core usage crashes earlier when i try to run it #140 ). With the default allocated 20GB of working memory it crashes after the: "2021-10-26 13:35:23 | INFO | Computing network" step.
I am running it in 'full output networks' mode, so i gues there is more text in the network to generate.

It gives a: "Worker exceeded 95% memory budget. Restarting" popup 3 times. After which the proces gets teminated.

Changing the variable of memory_limit in the command/network.py = "20GB" to "32GB" seems to fix it for now.
I think il try to reactivate the argparse variable for it in the develop branch soon (im not sure why it is commented out, but it makes sense to have it tweakable).

The underlying issue however is the high ammount of memory needed. The resulting network is 4.5 gb in size, but the fact that 20gb is not enough to compute (when running in single core mode) is a bit finicky. About this I'm not so sure how to fix it. The weird thing is also that before i had no issues generating these networks. Perhaps they were on the border and are slightly larger now i slightly changed my binding input. But it is a bit weird.

(I'm assigning this to myself untill ive reactivated the argparse variable in the develop branch, for the larger memory stuff I'm not asigning myself).

@JGASmits JGASmits self-assigned this Oct 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant