Fastest way to make python Object out of numpy array rows

You won't get great efficiency/speed gains as long as you are insisting on building Python objects. With that many items, you will be much better served by keeping the data in the numpy array. If you want nicer attribute access, you could cast the array as a record array (recarray), which would allow you to name the columns (as weight, height, etc) while still having the data in the numpy array.

dog_t = np.dtype([
    ('weight', int),
    ('height', int),
    ('width', int),
    ('girth', int)
])

dogs = np.array([
    (5, 100, 50, 80),
    (4, 80, 30, 70),
    (7, 120, 60, 90),
    (2, 50, 30, 50),
], dtype=dog_t)

dogs_recarray = dogs.view(np.recarray)

print(dogs_recarray.weight)
print(dogs_recarray[2].height)

You can also mix and match data types if you need to (if some columns are integer and others are float, for example). Be aware when playing with this code that the items in the dogs array needs to be specified in tuples (using ()) rather than in lists for the datatype to be applied properly.

Multiprocessing might be worth a look.

from multiprocessing import Pool dog_list = []

Function to append objects to the list:

def append_dog(i): dog_list.append(Dog(*dogs[i]))

Let multiple workers append to this list in parallel:

number_of_workers = 4 pool = Pool(processes=number_of_workers) pool.map_async(append_dog, range(len(dogs)))

Or as a shorter version:

from multiprocessing import Pool
number_of_workers = 4
pool = Pool(processes=number_of_workers)
pool.map_async(lambda i: dog_list.append(Dog(*dogs[i])), range(len(dogs)))

With a simple class:

class Foo():
    _id = 0
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z
        self.id = self._id
        Foo._id += 1
    def __repr__(self):
        return '<Foo %s>'%self.id


In [23]: arr = np.arange(12).reshape(4,3)

A straightforward list comprehension:

In [24]: [Foo(*xyz) for xyz in arr]
Out[24]: [<Foo 0>, <Foo 1>, <Foo 2>, <Foo 3>]

Default use of vectorize:

In [26]: np.vectorize(Foo)(*arr.T)
Out[26]: array([<Foo 5>, <Foo 6>, <Foo 7>, <Foo 8>], dtype=object)

Note that Foo 4 was skipped. vectorize performs a trial calculation to determine the return dtype (here object). (This has caused problems for other users.) We can get around that by specifying otypes. There's also a cache parameter that might work, but I haven't played with that.

In [27]: np.vectorize(Foo,otypes=[object])(*arr.T)
Out[27]: array([<Foo 9>, <Foo 10>, <Foo 11>, <Foo 12>], dtype=object)

Internally vectorize uses frompyfunc, which in this case works just as well, and in my experience is faster:

In [28]: np.frompyfunc(Foo, 3,1)(*arr.T)
Out[28]: array([<Foo 13>, <Foo 14>, <Foo 15>, <Foo 16>], dtype=object)

Normally vectorize/frompyfunc pass 'scalar' values to the function, iterating overall elements of a 2d array. But the use of *arr.T is a clever way of passing rows - effectively a 1d array of tuples.

In [31]: list(zip(*arr.T)) 
Out[31]: [(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 10, 11)]

Some comparative times:

In [32]: Foo._id=0
In [33]: timeit [Foo(*xyz) for xyz in arr]
14.2 µs ± 17.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [34]: Foo._id=0
In [35]: timeit np.vectorize(Foo,otypes=[object])(*arr.T)
44.9 µs ± 108 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [36]: Foo._id=0
In [37]: timeit np.frompyfunc(Foo, 3,1)(*arr.T)
15.6 µs ± 18.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

This is consistent with my past timings. vectorize is slow. frompyfunc is competitive with a list comprehension, sometimes even 2x faster. Wrapping the list comprehension in an array will slow it down, e.g. np.array([Foo(*xyz)...]).

And your original list comprehension:

In [40]: timeit [Foo(arr[i][0],arr[i][1],arr[i][2]) for i in range(len(arr))]
10.1 µs ± 80 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

That's even faster! So if your goal is a list rather than an array, I don't see the point to using numpy tools.

Of course these timings on a small example need to be viewed with caution.