Is there a more efficient way than using numpy.asarray() to generate an array from output in the form of a list?
This appears to be copying everything in memory, which doesn't seem like it would be that efficient with very large arrays.
(Updated) Example:
import numpy as np
a1 = np.array([1,2,3,4,5,6,7,8,9,10]) # pretend this has thousands of elements
a2 = np.array([3,7,8])
results = np.asarray([np.amax(np.where(a1 > element)) for element in a2])
I usually use np.fromiter:
results = np.fromiter((np.amax(np.amax(np.where(a1 > element)) for element in a2), dtype=int, count=len(a2))
You don't need to specify count but it allows numpy to preallocate the array. Here are some timings I did on https://www.pythonanywhere.com/try-ipython/:
In [8]: %timeit np.asarray([np.amax(np.where(a1 > element)) for element in a2])
1000 loops, best of 3: 161 us per loop
In [10]: %timeit np.frompyfunc(lambda element: np.amax(np.where(a1 > element)),1,1)(a2,out=np.empty_like(a2))
10000 loops, best of 3: 123 us per loop
In [13]: %timeit np.fromiter((np.amax(np.where(a1 > element)) for element in a2),dtype=int, count=len(a2))
10000 loops, best of 3: 111 us per loop
np.vectorize won't work the way you want, because it doesn't respect an out parameter. However, the lower-level np.frompyfunc will:
np.frompyfunc(lambda element: np.amax(np.where(a1 > element)),
1, 1)(a2, out=np.empty_like(a2))
frompyfunc. This looks great. Thanks!
elementis larger then all elements ina1, it is just the largest element ofa1. In any case the approach will scale very badly for largea1for this kind of function, so what are you actually doing here? Alsonp.frompyfuncdoesn't really speed up things. Note that the copying here should be very unimportant compared to the actual work done, trying to optimize things without knowing where your time is spend is usually a bad idea...