List comprehension for two variable loop in Python and numpy

Question

I have to create a 2D numpy array from the values x, y from a function return to plot using contourf from matplotlib, and so far I'm using a "C" like structure, that it seems to be very inefficient in Python:

    dim_x = np.linspace(self.min_x, self.max_x, self.step)
    dim_y = np.linspace(self.min_y, self.max_y, self.step)
    X, Y = np.meshgrid(dim_x, dim_y)

    len_x = len(dim_x)
    len_y = len(dim_y)


    a = np.zeros([len_x, len_y], dtype=complex)

    for i, y in enumerate(dim_y):
        for j, x in enumerate(dim_x):
            a[i][j] = aux_functions.final_potential(complex(x, y), element_list)

cs = plt.contourf(X, Y, (a.real), 100)

How can this be done in a more pythonic way?

Thanks!

What does your final_potential function do? What you want to do is "vectorize" that function, rather than writing nested loops. fromfunction or vectorize are fine, but they're just as inefficient as your loop. — Joe Kington
– Joe Kington, Commented Jun 13, 2012 at 15:00
Can you point me to some documentation about "vectorizing" my function? — Ivan
– Ivan, Commented Jun 13, 2012 at 17:24
Read through the various numpy tutorials. (e.g. tramy.us/numpybook.pdf Also see scipy-lectures.github.com ) The basic idea is to apply operations to the entire array instead of individual elements. If you'll give an example of your final_potential function, we can help walk you through it. It's usually quite simple, but it might not be immediately obvious if you haven't done it before. In some cases (e.g. finite difference methods) it simply can't be done, but these are relatively rare. — Joe Kington
– Joe Kington, Commented Jun 13, 2012 at 17:34
Also, as a quick example, consider a function that takes the sin of each item and if the result is greater than 0, multiplies it by 2, otherwise just returning sin(x). This could be written as result = np.sin(x); result[result > 0] *= 2, where x is an array of your values. — Joe Kington
– Joe Kington, Commented Jun 13, 2012 at 17:47

senderle · Accepted Answer · 2012-06-14 03:23:01Z

It would be ideal if you could rewrite final_potential as a vectorized function. A simple and, perhaps, too-obvious example:

>>> dim_x = np.linspace(0, 2, 5)
>>> dim_y = np.linspace(0, 2, 5)
>>> X * Y
array([[ 0.  ,  0.  ,  0.  ,  0.  ,  0.  ],
       [ 0.  ,  0.25,  0.5 ,  0.75,  1.  ],
       [ 0.  ,  0.5 ,  1.  ,  1.5 ,  2.  ],
       [ 0.  ,  0.75,  1.5 ,  2.25,  3.  ],
       [ 0.  ,  1.  ,  2.  ,  3.  ,  4.  ]])

But if you really can't do that, you could also vectorize:

>>> np.vectorize(lambda x, y: x * y + 2)(X, Y)
array([[ 2.  ,  2.  ,  2.  ,  2.  ,  2.  ],
       [ 2.  ,  2.25,  2.5 ,  2.75,  3.  ],
       [ 2.  ,  2.5 ,  3.  ,  3.5 ,  4.  ],
       [ 2.  ,  2.75,  3.5 ,  4.25,  5.  ],
       [ 2.  ,  3.  ,  4.  ,  5.  ,  6.  ]])

In your case, it would probably look something like this:

def wrapper(x, y): 
    return aux_functions.final_potential(complex(x, y), element_list)

a = np.vectorize(wrapper)(X, Y)

This will probably be a bit faster than nested for loops, although the overhead of a python function call will negate a lot of numpy's efficiency. In tests I've done in the past, using vectorize provided a modest 5x speedup. (That's compared to a 100x or 1000x speedup for pure numpy operations, as in the X * Y example.)

mgilson · Accepted Answer · 2012-06-13 14:50:01Z

1

I don't know if this would increase your performance at all, but there is numpy.fromfunction which could be used to construct your array a. Without a nice self-contained example, it will be a little tricky to timeit to see how difference the performance is however. e.g. how long does it take to run aux_functions.final_potential? If that function is expensive, it doesn't matter how much you optimize the loops around it.

Also, I'd imagine that a[i,j] is slightly more efficient than a[i][j], but I haven't tested it.

answered Jun 13, 2012 at 14:50

mgilson

312k70 gold badges656 silver badges722 bronze badges

1 Comment

jterrace Over a year ago

+1 for using fromfunction. It wouldn't increase performance, but that wasn't the question

Collectives™ on Stack Overflow

List comprehension for two variable loop in Python and numpy

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related