Faster alternative than a loop to create a numpy array

Question

I have an array which contains point clouds (about 100 ladar points). I need to create a set of numpy arrays as quickly as possible.

sweep = np.empty(
    shape=(len(sweep.points),),
    dtype=[
        ('point', np.float64, 3),
        ('intensity', np.float32),
        ## ..... more fields ....
    ]
)

for index, point in enumerate(sweep.points):
        sweep[index]['point'] = (point.x, point.y, point.z)
        sweep[index]['intensity'] = point.intensity
        ## ....more fields...

Writing an explicit loop is very inefficient and slow. Is there a better way to go about this?

I'm inferring that sweep is a list of points objects, where each point has attributes that you want to place fields of the array, right? Wait this won't work. After defining the empty array, sweep is now an array, not the list of objects! — hpaulj
– hpaulj, Commented Aug 16, 2018 at 16:45

kuppern87 · Accepted Answer · 2018-08-16 14:08:50Z

2

It's slightly faster to use a list comprehension to format the data and pass it directly to a numpy array:

np.array([((point.x, point.y, point.z), point.intensity)
          for point in points],
         dtype=[('point', np.float64, 3),
                ('intensity', np.float32)])

answered Aug 16, 2018 at 14:08

kuppern87

1,1359 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user2653663 Over a year ago

np.fromiter might be 5% faster still or so if that would matter. All it takes to change the code is to change np.array to np.fromiter, change the list comprehension to a generator expression, and add the argument count=len(points).

samol Over a year ago

Actually, Is there a parallel way to doing this? (Obviously without me explictly writing parallel code)

Collectives™ on Stack Overflow

Faster alternative than a loop to create a numpy array

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related