Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about Train environment #14

Open
YUHANG-Ma opened this issue Jun 5, 2022 · 5 comments
Open

Question about Train environment #14

YUHANG-Ma opened this issue Jun 5, 2022 · 5 comments

Comments

@YUHANG-Ma
Copy link

YUHANG-Ma commented Jun 5, 2022

Hi wenbo,
Thanks for your code sharing. I am trying to use your code to train my own model. Could I ask what typr of GPU and CPU are you based on when you are training your model? Like how many kernals and memory of cpu and GPUtpe(1080,2060 or 3090?)
Also, the type of my GPU is 3060 and it is not suit for your image env(cuda10). I couldn't run model.cuda( ) on my computer. So I changed the cudatoolkit to 11.0. In this way model.cuda() is sucessful but when I run build.sh, pg_op can't be built successfully. I gusess it is because the dismatch of cuda. Do you have any sugesstion on how to import pg_op on cuda11.0?
Looking forward to your reply : )

@wenbowen123
Copy link
Owner

Hi, we were using 2080Ti and the CPU memory was 32G. Did you replace the cuda version inside docker? If so, can you put the screenshot of the build error here? That'd help to check the problem.

@YUHANG-Ma
Copy link
Author

I just change the cuda env in python and not in docker. Here is the error when building.

Here is the cuda version in python. Cuda in docker is still 10.1.
01875927b03c09f6ff38cda5053fa3f
Here is the error when building.
2c6117057420076fd04e391b5b535ab

@wenbowen123
Copy link
Owner

wenbowen123 commented Jun 6, 2022

Outside of docker, there will be many overhead (more than just cuda related things) and thus it's hard to support. One possible solution is to replace the cuda version inside the docker, then you can still have all the other dependencies for the environments.

@waiyc
Copy link

waiyc commented Jul 1, 2022

@YUHANG-Ma I am having the same GPU 3060 as you and would like to upgrade the CUDA version in the docker to test this repo. Do you mind to tell me how you change the CUDA and cuDNN version in the docker container?

@wenbowen123
Copy link
Owner

@waiyc @YUHANG-Ma we just updated our docker for newer GPU, e.g. 3090
wenbowen123/catgrasp:cuda11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants