Buffer array operation

Question

How can I do operation on the array from the buffer?

import multiprocessing as mp
import numpy as np

# Initialization of the array

n_elem = 10
buffer = mp.RawArray('i', n_elem)
from_buffer = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
for _ in range(n_elem):
    from_buffer[_] = _
print(from_buffer)

>> [0 1 2 3 4 5 6 7 8 9]

Method that works to roll the array:

for _ in range(n_elem-1):
    from_buffer[_] = from_buffer[_ + 1]
from_buffer[-1] = 10

from_buffer2 = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
print(from_buffer2)

>> [ 1  2  3  4  5  6  7  8  9 10]

When from_buffer2 is "reinitialized", it stops "updating" buffer array:

from_buffer2 = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
from_buffer2 = np.roll(from_buffer2, -1, axis=0)
from_buffer2[-1] = 0
from_buffer = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
print(from_buffer)

>> [ 1  2  3  4  5  6  7  8  9 10]

I do not understand completely what is going on. When the array is generated from buffer, a change on from_buffer is visible on from_buffer2 as well when using index of the array.

Is there a way to manipulate the array differently?

yut23 · Accepted Answer · 2022-06-16 01:06:48Z

1

np.roll() returns a new array, so any changes to it won't be reflected in from_buffer. You can see this by looking at the "data pointer" line printed by np.info() (other output removed for clarity):

>>> buffer = mp.RawArray('i', n_elem)
>>> from_buffer = np.frombuffer(buffer, dtype=np.uint32)
>>> from_buffer[:] = np.arange(n_elem)
>>> np.info(from_buffer)
data pointer: 0x7f4e36ff0000

>>> from_buffer[:-1] = from_buffer[1:]; from_buffer[-1] = 10
>>> print(from_buffer)
[ 1  2  3  4  5  6  7  8  9 10]
>>> np.info(from_buffer)
data pointer: 0x7f4e36ff0000

>>> from_buffer2 = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
>>> np.info(from_buffer2)
data pointer: 0x7f4e36ff0000

>>> from_buffer2 = np.roll(from_buffer2, -1, axis=0)
>>> np.info(from_buffer2)
data pointer: 0x56109049db60

You can do what you want by assigning to the data in from_buffer2 instead of the name from_buffer2:

>>> from_buffer2 = np.reshape(np.frombuffer(buffer, dtype=np.uint32), -1)
>>> np.info(from_buffer2)
data pointer: 0x7f4e36ff0000

>>> from_buffer2[:] = np.roll(from_buffer2, -1, axis=0)
>>> print(from_buffer)
[ 2  3  4  5  6  7  8  9 10  1]

>>> from_buffer[:] = np.roll(from_buffer, -2, axis=0)
>>> print(from_buffer2)
[ 4  5  6  7  8  9 10  1  2  3]

answered Jun 16, 2022 at 1:06

yut23

3,09313 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mooder Over a year ago

Is it possible to manipulate the pointer?

yut23 Over a year ago

It looks like you can create a new array from a ctypes pointer: stackoverflow.com/questions/23930671

Collectives™ on Stack Overflow

Buffer array operation

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related