Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Will apex opt_level affect performance? #7

Open
SunDoge opened this issue Dec 25, 2020 · 2 comments
Open

Will apex opt_level affect performance? #7

SunDoge opened this issue Dec 25, 2020 · 2 comments

Comments

@SunDoge
Copy link

SunDoge commented Dec 25, 2020

I notice that you set the opt_level='O0', which is FP32 training instead of mixed-precision training. What would happen if using opt_level=O1 or higher opt_level?
@yaox12
Copy link
Owner

yaox12 commented Dec 27, 2020

I have tried opt_level=O1 and O2, they gave very close results but didn't show much speedup in training time.
I guess it is due to the communication bottleneck.

@SunDoge
Copy link
Author

SunDoge commented Dec 27, 2020

Thx.

@SunDoge SunDoge closed this as completed Dec 27, 2020
@yaox12 yaox12 mentioned this issue Jan 21, 2021
Open
@yaox12 yaox12 reopened this Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yaox12 @SunDoge