New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove fake all reduce step for onboard #118

Open

samsja opened this issue Oct 17, 2024 · 0 comments

Collaborator

samsja commented Oct 17, 2024 •

edited

Loading

We don't need a fake step for onboarding.

Only rank src needs to be aware that a new joiner is coming and need to potentially wait for it.

This can be done by creating its own process group for the sned/rcv.

This would fix #117 in a proper way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment