How to use xxhash in hashlib.file_digest?

Question

I want to quickly compute hash values of large files on disk to compare them to one another.

I'm using the following:

 import hashlib
 def sha256sum(filename):
    with open(filename, 'rb', buffering=0) as f:
        return hashlib.file_digest(f, 'sha256').hexdigest()

But I'd like to use xxhash since I hear it's faster. This doesn't work:

import hashlib
def xxhashsum(filename):
    with open(filename, 'rb', buffering=0) as f:
        return hashlib.file_digest(f, 'xxhash').hexdigest()

Is there a version that would?

YGA · Accepted Answer · 2024-08-07 17:21:00Z

0

Try like this:

import xxhash
import hashlib

def xxhashsum(filename, algo="xxh128"):
    if algo not in xxhash.algorithms_available:
        raise NotImplementedError
    digest = getattr(xxhash, algo)
    with open(filename, 'rb', buffering=0) as f:
        return hashlib.file_digest(f, digest).hexdigest()

edited Aug 7, 2024 at 17:21

YGA

10.1k16 gold badges53 silver badges54 bronze badges

answered Aug 2, 2024 at 22:34

wim

368k113 gold badges681 silver badges816 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

YGA Over a year ago

Perfect, thank you!

Collectives™ on Stack Overflow

How to use xxhash in hashlib.file_digest?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related