Fastest way to replace values in a numpy array with a list

Question

I want to read a list into a numpy array. This list is being replaced in every iteration of a loop and further operations are done on the array. These operations include element-wise subtraction from another numpy array for a distance measure, and checking a threshold condition in this distance using the numpy.all() function. Currently I am using np.array( list ) each time to convert the list to an array:

#!/usr/bin/python
import numpy as np
a = [1.33,2.555,3.444,5.666,4.555,6.777,8.888]
%timeit b = np.array(a)

100000 loops, best of 3: 4.83 us per loop

Is it possible to do anything better than this, if I know the size of the list and it is invariable? Even small improvements are welcome, as I run this a very large number of times.

I've tried %timeit(np.take(a,range(len(a)),out=b)) which takes much longer: 100000 loops, best of 3: 16.8 us per loop

In my opinion there is no faster way of converting a list to a numpy.array. I would rather try to vectorize your for loop. Can you be more specific what you do with the numpy.array ? — plonser
– plonser, Commented May 12, 2015 at 12:07
Where does the list come from? Can you use numpy earlier in your calculations to avoid creating a list? — Warren Weckesser
– Warren Weckesser, Commented May 12, 2015 at 12:54
@WarrenWeckesser No, I can't use numpy earlier in my calculation, this comes from a simulation engine, which I have little or no control over. — Sahil M
– Sahil M, Commented May 12, 2015 at 13:57
If the lists aren't that big, it might be faster to do everything with list operations. There is some overhead in creating an array. — hpaulj
– hpaulj, Commented May 12, 2015 at 15:47

Lee · Accepted Answer · 2015-05-12 13:06:43Z

1

As you "know the size of the list and it is invariable", you can set up an array first:

b = np.zeros((7,))

This then works faster:

%timeit b[:] = a
1000000 loops, best of 3: 1.41 µs per loop

vs

%timeit b = np.array(a)
1000000 loops, best of 3: 1.67 µs per loop

answered May 12, 2015 at 13:06

Lee

31.4k31 gold badges124 silver badges187 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Sahil M Over a year ago

Thanks, this is marginally better. If there is nothing better than this, I'll accept it as an answer,

Sahil M Over a year ago

I ended up considerably speeding up by using hpaulj's suggestion of not using numpy arrays at all. However this is still the correct and best answer to the above question, thanks!

Collectives™ on Stack Overflow

Fastest way to replace values in a numpy array with a list

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related