structured array of floats and int to array of floats

Question

I am importing a csv file

data = np.genfromtxt('na.csv', delimiter=",", dtype=[('latitude', 'f8'), ('longitude', 'f8'), ('location_id','i4'), ('location_name', 'S60'), ('location_group_id', 'i4'), ('location_group_name', 'S32')])

and considering rows by location_group_ids, one by one.

l_g_id_set = set()
l_g_id_set.update(data['location_group_id'])

for lgid in l_g_id_set:
    # rows with location group id == lgid
    group = data[data['location_group_id']==lgid]

So far, I only included latitude and longitude, which are two float values in the 0th and 1st position of the structured array from the csv file.

    # structured array of latitude-longitude
    latlon = group[list(group.dtype.names[:2])]

    # convert the structured array into numpy array of floats
    llarray = latlon.view((float, len(latlon.dtype.names)))

Now I want to include location_id, which is an integer value in the 2nd position of the array, to latlon and llarray. Rather than making this another structured array, I'd want llarray a 2D float array with 3 columns for ease of calculation.

However when I try the following, only changing 2 to 3

    # structured array of latitude-longitude
    latlon = group[list(group.dtype.names[:3])]

    # convert the structured array into numpy array of floats
    llarray = latlon.view((float, len(latlon.dtype.names)))

it fails, throwing the following error.

    llarray = latlon.view((float, len(latlon.dtype.names)))
ValueError: new type not compatible with array.

How can I fix this, and why is my fix failing?

The problem is, I think, with trying to 'view' the int data as float - without copying or overwriting. np.ones((3,),dtype=int).view(float) produces the same error. — hpaulj
– hpaulj, Commented Mar 7, 2014 at 21:36

hpaulj · Accepted Answer · 2016-04-06 02:37:30Z

1

This transformation works

dtype1=[('latitude', 'f8'), ('longitude', 'f8'), ('location_id', 'f4')]
data1=data[list(data.dtype.names[:3])].astype(dtype1)

But data1.view(float) still gives the error

dtype2=[('latitude', 'f8'), ('longitude', 'f8'), ('location_id', 'f8')]
data2=data[list(data.dtype.names[:3])].astype(dtype2)
data2.view(float).reshape(-1,3)
data2.view((float,3))   # equivalent view

is ok.

Sample data:

In [211]: data[:3]
Out[211]: 
array([(1.2, 2.3, 100, 'testing', 45, 'another'),
       (1.2, 2.3, 200, 'testings', 45, 'xxx'),
       (1.2, 2.3343, 300, 'testings', 45, 'xxx')], 
      dtype=[('latitude', '<f8'), ('longitude', '<f8'), ('location_id', '<i4'), ('location_name', 'S60'), ('location_group_id', '<i4'), ('location_group_name', 'S32')])

In [212]: data2[:3].view(np.float).reshape(-1,3)
Out[212]: 
array([[   1.2   ,    2.3   ,  100.    ],
       [   1.2   ,    2.3   ,  200.    ],
       [   1.2   ,    2.3343,  300.    ]])

In [230]: data2.view(np.float).reshape(-1,3).max(axis=0)
Out[230]: array([   1.2   ,    2.3343,  300.    ])
In [234]: data2['longitude'].max()
Out[234]: 2.3342999999999998
In [236]: data2.view(np.float).reshape(-1,3)[:,1].max()
Out[236]: 2.3342999999999998

edited Apr 6, 2016 at 2:37

answered Mar 7, 2014 at 21:49

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Eric Over a year ago

Thank you. A quick question: how do you get the minimum of 0th and 1st column(latitude and longitude) in data2? min(data2[:,0]) and min(data2[:,1]) does not work, strangely.

Eric Over a year ago

yeah. By min I meant np.min

hpaulj Over a year ago

Are you taking the min after the view and reshape? I've added several examples of taking max values.

kzorro · Accepted Answer · 2014-03-07 19:37:29Z

0

Hmm. Maybe you will have luck with this.

f_latlon = latlon.astype(np.float)

answered Mar 7, 2014 at 19:37

kzorro

1,6922 gold badges11 silver badges4 bronze badges

Collectives™ on Stack Overflow

structured array of floats and int to array of floats

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related