For image processing, I need an array (array type, not numpy array) for 2 million 32 bit words. If I use something like:
tb = array.array( 'i', ,(0,)*2000000)
it requires 126 msec. It's large and I don't even need to initialize the array. I don't know Python internals but I assume that the statement generate tons on malloc() (memory allocator) and free() (memory deallocator).
Is there another way to create a very large Python array?
This is much faster, because it doesn't create a long, temporary tuple:
tb = array.array('i', (0,)) * 2000000
.frombytes(b'\0' * 8_000_000). Cool!This does the same thing, but "should" run at least 10 times faster, by avoiding the needless expense of creating, and crawling over, a multi-million element tuple of unbounded Python ints:
>>> tb = array.array('i')
>>> tb.frombytes(b'\0' * 8_000_000)
>>> len(tb)
2000000
>>> all(i == 0 for i in tb)
True
Note: I'm assuming you're running on a platform where the array typecode i denotes a 4-byte integer type (that's why I changed your 2 million to 8 million). That's very likely, but if you're not sure of that, then slightly fancier code is needed:
>>> tb = array('i')
>>> tb.frombytes(b'\0' * (2_000_000 * tb.itemsize))
Of course tb.itemsize there returns 4 on my box.
