1

Summary

I have copied (rsync --archive) a folder from an ext4 filesystem to a zfs file system with compression on. Now, I'm trying to verify that both folders are identical so that I can safely delete the source folder.

When re-running rsync, no additional bytes are transferred. So, rsync is convinced that both folders are identical.

However, using du, du -b, or md5sum yield different results for both folders.

How can I convince myself that both folders are identical before deleting the source folder?

Examples

I've uploaded a test folder 883 containing four files.

fiedl@ext4 ▶ du 883
20  883
fiedl@zfs ▶ du 883
57  883

fiedl@ext4 ▶ du -s 883
20  883
fiedl@zfs ▶ du -s 883
57  883

fiedl@ext4 ▶ du -sb 883
4660    883
fiedl@zfs ▶ du -sb 883
570 883

fiedl@ext4 ▶ du 883/*
4   883/big_image001.gif
4   883/image001.gif
4   883/medium_image001.gif
4   883/thumb.png
fiedl@zfs ▶ du 883/*
10  883/big_image001.gif
10  883/image001.gif
10  883/medium_image001.gif
10  883/thumb.png

fiedl@ext4 ▶ tar -cf - 883 | md5sum
7c8a4ff31fdf594b04173789b23c7bb8  -
fiedl@zfs ▶ tar -cf - 883 | md5sum
f207dbadd75126665af300705774c97f  -
Improve this question
3
  • Can you please show some examples of how the output is different? Also, can you please provide a sample file so we can test it ourselves on our own ext4 and zfs installations? Commented Feb 3, 2021 at 21:02
  • 1
    Thanks, cryptarch, for your suggestion. I've added some examples. In the meantime, I may have found a workaround using a combination of find, md5sum, sort, and diff. Should I post this as an answer or should I wait for someone posting a proper way to do it? Commented Feb 3, 2021 at 22:18
  • If you think you have an answer, I reckon you should post it :) Commented Feb 3, 2021 at 22:31

1 Answer 1

Reset to default
2

md5sum and diff

Under the assumption that the observed differences are the results of different meta data and zfs compression, the respective md5sum of the individual files should still be the same.

cd /path/to/ext4 && find . | xargs md5sum | sort > ~/md5sum-index-ext4
cd /path/to/zfs  && find . | xargs md5sum | sort > ~/md5sum-index-zfs
diff ~/md5sum-index-ext4 ~/md5sum-index-zfs

This workaround recursively lists all files within the directories using find, adds the md5sum for each file, sorts the results because find may return the contents in different order. Then the results for both folders can be compared. If they are identical, the diff is empty. Otherwise, the files with different binary content will show up in the diff.

diff -r

I was able to compare smaller folders directly using diff -r. However, this has crashed for large folders, maybe due to the lack of memory.

diff -r /path/to/ext4 /path/to/zfs

If both folders are identical, the diff is empty.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.