Python ctype struct with array of structs

Question

I have some struct, with dynamic entries count. I receiving bytearray from UDP and parsing this message as follows:

class MsgStruct(Structure):
    _pack_ = 1

    def __init__(self, data=None):
        if data:
            self.unpack(data)

    def unpack(self, raw):
        fit = sizeof(self)
        memmove(addressof(self), raw[:fit], fit)

    def pack(self):
        return bytearray(self)[:]

    def size(self):
        return sizeof(self)


class MessageEntry(MsgStruct):

    _fields_ = [
        ('type', c_byte),
        ('flag', c_byte),
        ('count', c_int)]


class Message(MsgStruct):

    _fields_ = [
        ('id', c_int),
        ('entry_count', c_int)]

    entries = []

    def __init__(self, data=None):
        MsgStruct.__init__(self, data=data)
        if data:
            self.parseEntries(data[self.entry_count:])

    def parseEntries(self, data):
        offset = 0
        size = sizeof(MessageEntry())
        for count in range(self.entry_count):
            entry = MessageEntry(data[offset:offset+size])
            self.entries.append(entry)
            offset += size

but I think there is a better way to parsing the message using ctypes.Array or POINTER and trying this:

class Message(MsgStruct):

    _fields_ = [
        ('id', c_int),
        ('entry_count', c_int),
        ('entries', POINTER(MessageEntry))]

    def __init__(self, data=None):
        MsgStruct.__init__(self, data=data)
        if data:
            self.parseEntries(data[self.entry_count:])

    def parseEntries(self, data):
        offset = 0
        size = sizeof(MessageEntry())
        elems = (MessageEntry * self.entry_count)()
        self.entries = cast(elems, POINTER(MessageEntry))
        for count in range(self.entry_count):
            self.entries[count] = MessageEntry(data[offset:offset+size])
            offset += size

But when I try to print entries, i'm fall into an endless cycle

msg = Message(x)
for i in msg.entries:
    print(i)

What wrong i doing? Is there another way to parsing message with dynamic entries?

CristiFati · Accepted Answer · 2019-01-14 18:12:46Z

I want to start with the note that I don't see where is the entry_count attribute initialized.

Iterating over a pointer as if it was a sized array is conceptually wrong (as also pointed out in [Python 3]: ctypes - A foreign function library for Python). In C, it's possible to go beyond an array bounds, but ctypes forbids it.
Here's a simpler example that uses ctypes.c_char as the base type (MessageEntry correspondent).

code.py:

#!/usr/bin/env python3

import sys
import ctypes


def main():
    CharArr5 = ctypes.c_char * 5
    b5 = b"12345"
    ca5 = CharArr5(*b5)
    print("Print array ...")
    for c in ca5:
        print(c)

    cp = ctypes.cast(ca5, ctypes.POINTER(ctypes.c_char))
    max_values = 10
    print("\nPrint pointer (max {:d} values) ...".format(max_values))
    for idx, c in enumerate(cp):
        print(c)
        if idx >= max_values:
            print("Max value number reached.")
            break


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Output:

(py_064_03.06.08_test0) e:\Work\Dev\StackOverflow\q054178876>"e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32

Print array ...
b'1'
b'2'
b'3'
b'4'
b'5'

Print pointer (max 10 values) ...
b'1'
b'2'
b'3'
b'4'
b'5'
b'\x00'
b'\x00'
b'\x00'
b'\x00'
b'\x00'
b'\x00'
Max value number reached.

As seen, it is possible to iterate over the pointer, but the loop never ends (it will end when reaching an unavailable / invalid memory address, and the program will segfault (Access Violation)).

Assuming that entry_count is properly initialized (if not, make sure to initialize it), use it to keep the loop inside bounds (as shown below):

for idx in range(msg.entry_count):
    msg.entries[idx]  # Do smth with it
    # ...

for idx, entry in enumerate(msg.entries):
    if idx >= msg.entry_count:
        break
    entry  # Do smth with it
    # ...

Or you could use one of the above to implement Iterator Protocol for Message.

Collectives™ on Stack Overflow

Python ctype struct with array of structs

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related