Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rsync: Modeled after fsspec.generic.rsync this uses two fs explicitly #1403

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Commits on Oct 24, 2023

  1. Modeled after fsspec.generic.rsync this uses two fs explicitly

    Pretty much identical to the generic one but doesn't create the
    directories async. The generic one defines a make_many_dirs which
    tries to make them async if it can and falls back to serial, this just
    uses the simpler version of non async. Can do the same helper function
    though.
    
    This is needed for cases of syncing two different storage systems that
    might have the same protocol. An example would be two different s3
    buckets that use different credentials. This allows you to make two
    different s3fs instances each passing different parameters and still
    sync them.
    
    The generic interface is great for giving a simple implicit way to
    sync things, but often we need fine grained control of what our
    filesystem parameters are so having an explicit method as well is
    useful. (Zen of Python line 2)
    f4hy committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    156538e View commit details
    Browse the repository at this point in the history