Convert elements of multi-dimensional numpy array to float32

Question

I have a complicated nested numpy array which contains list. I am trying to converted the elements to float32. However, it gives me following error:

ValueError                                Traceback (most recent call last)
<ipython-input-225-22d2824961c2> in <module>
----> 1 x_train_single.astype(np.float32)

ValueError: setting an array element with a sequence.

Here is the code and sample input:

    x_train_single.astype(np.float32)

    array([[ list([[[0, 0, 0, 0, 0, 0]], [-1.0], [0]]),
    list([[[0, 0, 0, 0, 0, 0], [173, 8, 172, 0, 0, 0]], [-1.0], [0]])
   ]])

Do you need to keep that peculiar array structure or is a plain 4D float32 array acceptable? — Paul Panzer
– Paul Panzer, Commented Oct 22, 2019 at 6:29
You do realize that a numpy.float32 inside a Python list takes more space than an ordinary Python float? — Paul Panzer
– Paul Panzer, Commented Oct 22, 2019 at 6:31
Yes, actually it is a requirement to train my model in tensor flow. I have updated the sample data to have more understanding — user1584253
– user1584253, Commented Oct 22, 2019 at 6:33
Tensorflow works with numpy arrays, you don't need it in list unless you have your own data transformers on the data. — mujjiga
– mujjiga, Commented Oct 22, 2019 at 6:34

Paul Panzer · Accepted Answer · 2019-10-22 07:08:06Z

As your array contains lists of different sizes and nesting depths, I doubt that there is a simple or fast solution.

Here is a "get-the-job-done-no-matter-what" approach. It comes in two flavors. One creates arrays for leaves, the other one lists.

>>> a
array([[list([[[0, 0, 0, 0, 0, 0]], [-1.0], [0]]),
        list([[[0, 0, 0, 0, 0, 0], [173, 8, 172, 0, 0, 0]], [-1.0], [0]])]],
      dtype=object)

>>> def mkarr(a):
...     try:
...         return np.array(a,np.float32)
...     except:
...         return [*map(mkarr,a)]
... 
>>> def mklst(a):
...     try:
...         return [*map(mklst,a)]
...     except:
...         return np.float32(a)
... 

>>> np.frompyfunc(mkarr,1,1)(a)
array([[list([array([[0., 0., 0., 0., 0., 0.]], dtype=float32), array([-1.], dtype=float32), array([0.], dtype=float32)]),
        list([array([[  0.,   0.,   0.,   0.,   0.,   0.],
       [173.,   8., 172.,   0.,   0.,   0.]], dtype=float32), array([-1.], dtype=float32), array([0.], dtype=float32)])]],
      dtype=object)

>>> np.frompyfunc(mklst,1,1)(a)
array([[list([[[0.0, 0.0, 0.0, 0.0, 0.0, 0.0]], [-1.0], [0.0]]),
        list([[[0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [173.0, 8.0, 172.0, 0.0, 0.0, 0.0]], [-1.0], [0.0]])]],
      dtype=object)

mujjiga · Accepted Answer · 2019-10-22 06:30:30Z

0

if number of columns is fixed then

np.array([l.astype(np.float) for l in x_train_single.squeeze()])

But it will remove the redundant dimensions, convert everything into numpy array.

Before: (1, 1, 1, 11, 6)

After: (11,6)

answered Oct 22, 2019 at 6:30

mujjiga

17.1k2 gold badges37 silver badges54 bronze badges

Comments

hpaulj · Accepted Answer · 2019-10-22 06:44:31Z

Try this:

np.array(x_train_single.tolist())

Looks like you have a (1,1) shaped array, where the single element is a list. And the sublists a consistent in size.

I expect you will get an array with shape (1, 1, 1, 11, 6) and int dtype.

or:

np.array(x_train_single[0,0])

Again this extracts the list from the array, and then makes an array from that.

My answer so far was based on the display:

array([[list([[[173, 8, 172, 0, 0, 0], [512, 58, 57, 0, 0, 0],  
     ...: [513, 514, 0, 0, 0, 0], [515, 189, 516, 0, 0, 0], [309, 266, 0, 0, 0, 
     ...: 0],  
     ...: [32, 310, 0, 0, 0, 0], [271, 58, 517, 0, 0, 0], [164, 40, 0, 0, 0, 0],
     ...:  [38, 32, 60, 0, 0, 0], [38, 83, 60, 0, 0, 0], [149, 311, 0, 0, 0, 0]]
     ...: ])]])

The new display is more complicated

array([[ list([[[0, 0, 0, 0, 0, 0]], [-1.0], [0]]), 
     ...:     list([[[0, 0, 0, 0, 0, 0], [173, 8, 172, 0, 0, 0]], [-1.0], [0]])]])

because the inner lists differ in size. It can't be made into a numeric dtype array.

It can be turned into a (1,2,3) shape array, but still object dtype with 1d list elements.

Collectives™ on Stack Overflow

Convert elements of multi-dimensional numpy array to float32

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related