Convert string to ndarray in python

Question

I am reading a stream of data from an A-D converter via a socket from Python; the data come in as raw bytes. I want to format these bytes as int32 and place them into an ndarray. The read process looks something like this:

def datarecv():
    global msgbuf
    binlen = BURSTLEN + 4
    while len(msgbuf) < binlen:
        msgbuf = msgbuf + socket.recv(4096)
    reply = msgbuf[0:binlen]
    msgbuf = msgbuf[binlen:]
    # each recv comes with a 4 byte header that I throw away...
    return reply[4:]

The following is used successfully to write the received data to a file:

with open(filename, "wb') as f:
    bytesremaining = framesize
    for i in range(lines):
        f.write(datarecv()[0:min(linesize, bytesremaining)])
        bytesremaining -= linesize

I can then read back the file with something like this:

>>> data = numpy.fromfile(filename, dtype='int32')
>>> type(data)
<type 'numpy.ndarray'>

So my data variable is the format I'm looking for, I.E.

>>> data[1:10]
array([4214234234, 2342342342, 2342342342, 34534535, 345345353, 5675757, 
      2142423424, 35334535, 35353535, 4754745754], dtype=int32)

** BUT ** I want to omit the intermediate step of writing to a file. After I read in the raw stream of data I want to make it an ndarray so that I can manipulate the data. I can change the line from

        f.write(datarecv()[0:min(linesize, bytesremaining)])

to

        bigbuf = bigbuf + datarecv()[0:min(linesize, bytesremaining)]

and then I end up with a big string. It's a string of raw bytes (not ASCII) which I have to convert to 32 bit integers. I'm hung up on this last step. I hope this makes sense what I'm asking. Thanks.

Warren Weckesser · Accepted Answer · 2014-10-03 03:15:21Z

1

You can convert bigbuf to an array with numpy.fromstring

For example:

In [21]: bigbuf = "\1\0\0\0\2\0\0\0"

In [22]: fromstring(bigbuf, dtype=np.int32)
Out[22]: array([1, 2], dtype=int32)

edited Oct 3, 2014 at 3:15

answered Oct 3, 2014 at 3:08

Warren Weckesser

116k20 gold badges207 silver badges224 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Convert string to ndarray in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related