Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat fsdp diloco #29

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Feat fsdp diloco #29

wants to merge 12 commits into from

Commits on Sep 10, 2024

  1. wip add pure torch fsdp

    @samsja
    samsja committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    75ede2a View commit details
    Browse the repository at this point in the history

  2. add something somewhat working

    @samsja
    samsja committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    ee89d33 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. add wandb and real data

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    c217dd4 View commit details
    Browse the repository at this point in the history

  2. fix data

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    94bc0ae View commit details
    Browse the repository at this point in the history

  3. do all reduce on cpu

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    5b701d7 View commit details
    Browse the repository at this point in the history

  4. do all reduce on cpu

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    04dd585 View commit details
    Browse the repository at this point in the history

  5. fix opt step on cpu

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    e926e6c View commit details
    Browse the repository at this point in the history

  6. fix opt step on cpu

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9f680f2 View commit details
    Browse the repository at this point in the history

  7. fix batch size stuff

    @samsja
    samsja committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    8576f1c View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. add helper script

    @samsja
    samsja committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    71a9f89 View commit details
    Browse the repository at this point in the history

  2. fix it

    @samsja
    samsja committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    88b5250 View commit details
    Browse the repository at this point in the history

  3. fix it

    @samsja
    samsja committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    5957cae View commit details
    Browse the repository at this point in the history