2

Assume an rsync --recursive --ignore-existing foo bar copy command was being run for a large directory tree named foo, but that that command got prematurely interrupted. For example, because of a sudden power failure on the target machine.

Running the above command again can save a lot of time for any files/dirs already fully copied to the destination bar. But what about any file(s) that was right in the middle of being copied? Presumably, that file exists at the destination in an incomplete/broken state? Having it simply ignored as a "file that's already done" would be very undesirable, of course.

Does rsync handle this (extremely run of the mill) scenario intelligently without being directed to do so? (If so, how?) Or does it only do so if specifically instructed by means of more or different flags?

Frustratingly, neither the man page nor any vaguely related questions or answers on this site seem to address this very important detail in a clear way. (Sorry if I just didn't look very well.)

Improve this question

1 Answer 1

Reset to default
2

You can use the -c option to make rsync skip files by calculating and comparing their checksum instead of date and size:

-c, --checksum

skip based on checksum, not mod-time & size

That should copy all files again that doesn’t have the right contents.

You also have to remove the--ignore-existing flag, otherwise files that already exists on the destination will not be transferred.

Improve this answer
8
  • Are you saying/implying that the --ignore-existing flag causes both modification time and size in bytes to be the criteria used to determine whether to skip? In that case, shouldn't that size check be sufficient as is? Or is there a scenario conceivable where the full exact amount of bytes were copied, but somehow became garbled nonetheless? Commented Jan 28, 2024 at 7:58
  • 1
    Maybe a good idea to include in this answer the full man text for the checksum flag rather than the abbreviated one liner summary. "--checksum, -c This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a "quick check" that (by default) checks if each file's size and time of last modification match between the sender and receiver." (emphasis mine). In any case, thanks for nudging me in the right direction. Commented Jan 28, 2024 at 8:08
  • One more thing: should the -c flag be used instead of the --ignore-existing flag or in addition to that flag? (Maybe I'm just pedantic, but I feel that very explicit wording should be used when describing a detail this import.) Commented Jan 28, 2024 at 8:24
  • 1
    If you include the flag it won’t update any file that already exists on the target regardless of the state of that file. Commented Jan 28, 2024 at 9:01
  • 2
    So long story short: I conclude that in the majority of cases, one should specify neither --ignore-existing (due to complete lack of detection of file corruption) nor --checksum (because it's damn slow when dealing with large files on slow-ish volumes). Commented Jan 28, 2024 at 9:12

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.