assert equal for numpy object

Question

I have 2 npz files I'd like to compare through an assert from numpy testing module:

https://numpy.org/doc/stable/reference/routines.testing.html

From documentation I understood that .npz files are loaded as instance :

https://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.load.html https://www.kite.com/python/docs/numpy.lib.npyio.NpzFile

From my understanding I thought .npz files were dict but they are loaded as instance and I end up with this error :

AssertionError: 
Items are not equal:
 ACTUAL: <numpy.lib.npyio.NpzFile object at 0x11d96c690>
 DESIRED: <numpy.lib.npyio.NpzFile object at 0x11d97a5d0>

here is my code :

import numpy as np

_npz_test = np.load(npz_test, allow_pickle=True)
_npz_res = np.load(npz_res, allow_pickle=True)
np.testing.assert_equal(_npz_test, _npz_res)

Those .npz files are composed of 3 different files to which I can access when printing:

_npz_test.files

I suppose I could use a workaround by iterating through the keys but I'd like a one line solution to do this.

I can share the files if needed

Assert equal expects array-like objects, opened npz files are not arrays. You can either iterate over the keys, or open the files as byte arrays and compare them. — Dinari
– Dinari, Commented Sep 16, 2020 at 14:26
is there any way to make it as something on which I could apply an assert ? — D Prat
– D Prat, Commented Sep 16, 2020 at 14:27
[np.testing.assert_equal(_npz_test[x],_npz_res[x]) for x in _npz_test.keys()] will throw assert error if they are not equal, also you need to verify that all the keys exists in both objects — Dinari
– Dinari, Commented Sep 16, 2020 at 14:30

senderle · Accepted Answer · 2020-09-16 17:47:55Z

2

While assert_equal is able to handle numpy objects inside simple Python containers like dict and list, it's not flexible enough to handle more generic containers. And the object returned by load is not actually a dictionary or a subclass of a dictionary. It's just a "dictionary-like object," so assert_equal doesn't know what to do with it.

Fortunately this is quite easy to handle. We can explicitly convert the objects to dictionaries:

>>> import numpy
>>> a = numpy.array([1, 2, 3])
>>> b = numpy.array([4, 5, 6])
>>> numpy.savez_compressed('foo.npz', a=a, b=b)
>>> numpy.savez_compressed('bar.npz', a=a, b=b)
>>> foo = numpy.load('foo.npz')
>>> bar = numpy.load('bar.npz')
>>> numpy.testing.assert_equal(dict(foo), dict(bar))

I'm not sure this will work work for all corner cases, but it's likely to work most of the time.

answered Sep 16, 2020 at 17:47

senderle

152k36 gold badges218 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

D Prat Over a year ago

I didn't thought of doing it this way ! But it's funny it's not working when transforming it to a dict using dict(), but it's working when transforming it to a list using list(), at least for me... So I propose you to edit your post and replace dict with list. Thank you !

senderle Over a year ago

@DPrat, could you say more then about how the .npz files are saved? It doesn't really make sense to me that list would work while dict would not. I want to make sure I understand what's going on so that the answer is as correct as possible.

senderle Over a year ago

@DPrat after looking at a couple of things, I'm concerned that using list is producing incorrect results. If you call list on a dict-like object, you get a sequence of keys — just the keys. So I'm concerned that it's not checking for equality of arrays, it's just checking that the two objects have the same keys.

D Prat Over a year ago

indeed you were right, using list was returning a list of keys. I had to workaround this by iterating over the keys of the numpy object and converting each to a dict

Collectives™ on Stack Overflow

assert equal for numpy object

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related