Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Detect and propagate renames/moves instead of delete+create #576

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Detect and propagate renames/moves instead of delete+create #576

wants to merge 3 commits into from

Commits on May 3, 2023

  1. Use Update.archive as Common.prevState 

    The primary intention of this change is to have more information
available about the previous archive in the updateItem. As a bonus,
this means less allocations and less memory consumption. This gain is
unfortunately lost in network transfer.

During network transfer, 0-8 bytes are added for each file change and up
to [file name length + ArchiveFile length (in range of roughly 57 to 137
bytes but can be considerably more)] bytes are added per each child,
recursively, for a directory deletion or with any change to any child
within the directory. When received over a network, in memory this data
structure requires for each child [name length + 202 to 282, and may be
considerably more] bytes. For example, deleting a largish directory of
100 000 files (regardless of depth of hierarchy) may easily add 10 MB to
the network transfer and require additional 25-35 MB of memory on the
receiving end. (Note that these sizes are not fixed, are counted _very_
roughly here, and can change from version to version.)
    @tleedjarv
    tleedjarv committed May 3, 2023
    Configuration menu
    Copy the full SHA
    097c1d9 View commit details
    Browse the repository at this point in the history

  2. Detect and propagate renames/moves 

    A rename or a move is currently seen as a delete on the old path and a
create on the new path. This means that propagating what was from
users's perspective a simple rename can require copying gigabytes of
data for big files and directories (this is in best case scenario when
local copy shortcut is used instead of transmitting those gigabytes over
a network).

Add new functionality that enables detecting renames and moves, and
propagating them without copying any data. If this is not possible (due
to conflicts, errors or user actions) then it falls back to copying, as
before. Add a new user preference to control this (defaults to "off").

This is just a shortcut (akin to copying locally instead of transmitting
over a network).

Renames/moves are detected for files and directories, and only if the
contents have not changed (for directories, "contents" means the names
and contents of all its children, recursively).
    @tleedjarv
    tleedjarv committed May 3, 2023
    Configuration menu
    Copy the full SHA
    71a941d View commit details
    Browse the repository at this point in the history

  3. Regen strings.ml

    @tleedjarv
    tleedjarv committed May 3, 2023
    Configuration menu
    Copy the full SHA
    efaaf9e View commit details
    Browse the repository at this point in the history