2

I'm writing a callback function in Python:

@ctypes.CFUNCTYPE(None, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, np.ctypeslib.ndpointer(dtype=ctypes.c_ushort))
def FastModeCallback(hCam, x, y, w, h, binx, biny, imageBuffer_ptr):

    image = np.ctypeslib.as_array(imageBuffer_ptr, shape=(h, w))

How can I convert imageBuffer_ptr into a Numpy array of shape=(h,w) and dtype=c.ushort?

I read this answer: Convert Pointer to Array of Unsigned Chars to Numpy Array

and this answer: Getting data from ctypes array into numpy

Both advise using np.ctypeslib.as_array but when I try it I get this error:

Exception ignored on calling ctypes callback function: <function FastModeCallback at 0x00000224E2992DC0>
Traceback (most recent call last):
  File "C:\Users\Dave\data\Code\Python\Cheese\atik.py", line 472, in FastModeCallback
    image = np.ctypeslib.as_array(imageBuffer_ptr, shape=(h, w))
  File "C:\Program_Files\anaconda3\lib\site-packages\numpy\ctypeslib.py", line 521, in as_array
    return array(obj, copy=False)
ValueError: '<P' is not a valid PEP 3118 buffer format string

Google makes it sound like this is a long-standing bug in Python (I'm on 3.8) - what is a workaround?

3
  • Maybe stackoverflow.com/questions/58049957/… could help. Commented Apr 29, 2021 at 22:29
  • Also how are you calling the function? And what arguments are you passing to it? Commented Apr 29, 2021 at 22:39
  • @CristiFati It's a 3rd party DLL; I don't have the source and don't know how it's called. Commented Apr 30, 2021 at 16:30

1 Answer 1

3

Listing [Python.Docs]: ctypes - A foreign function library for Python and [NumPy]: C-Types Foreign Function Interface (numpy.ctypeslib). The latter states about the following 2 items (emphasis is mine):

  1. as_array

    Create a numpy array from a ctypes array or POINTER.

  2. ndpointer

    An ndpointer instance is used to describe an ndarray in restypes and argtypes specifications. This approach is more flexible than using, for example, POINTER(c_double), since several restrictions can be specified, which are verified upon calling the ctypes function.

imageBuffer_ptr is not a "ctypes array or POINTER" (literally: it's not a ctypes.POINTER instance, but ctypes.c_void_p one (there's a subtle difference between the 2)).

The C layer (which in our case lies in the middle) doesn't know about NumPy arrays and so on, so from its PoV, that argument is simply an unsigned short* (doesn't have information about shape, ...). So, on it's other side (in the Python callback), you have to reconstruct the array from the plain C pointer. That can be achieved in 2 ways, like I did in the example below:

  • callback_ct: Use good-old C way: modify the callback's signature to use ct.POINTER(ct.c_ushort). That is the most straightforward, but the drawback is that it loses all the extra checks when called, meaning that it takes any pointer not just a ndarray one

  • callback_np: Manually recreate the array from the argument (using a series of conversions). It's more advanced (some might even consider it a bit hackish)

code00.py:

#!/usr/bin/env python

import sys
import ctypes as ct
import numpy as np


Base = ct.c_ushort

NpPtr = np.ctypeslib.ndpointer(
    dtype=Base,
    flags='C_CONTIGUOUS',
    #ndim=1,
)

CtPtr = ct.POINTER(Base)


@ct.CFUNCTYPE(None, ct.c_int, ct.c_int, NpPtr)
def callback_np(w, h, data):
    print("\ncallback_np:")
    #print(data, type(data), type(data).__mro__, hex(data.value))
    #print(data._ndim_, data._dtype_, data._shape_, data._type_, data._objects)
    img = np.ctypeslib.as_array(ct.POINTER(Base).from_address(ct.addressof(data)), shape=(h, w))
    #img = np.ctypeslib.as_array(data, shape=(h, w))  # Original form
    print(img)


@ct.CFUNCTYPE(None, ct.c_int, ct.c_int, CtPtr)
def callback_ct(w, h, data):
    print("\ncallback_ct:")
    img = np.ctypeslib.as_array(data, shape=(h, w))
    print(img)


def main(*argv):
    w = 2
    h = 3
    arr = np.array(range(h * w), dtype=Base)
    print("Array argument: ", arr, type(arr), arr.shape, arr.dtype, arr.ndim)
    callback_ct(w, h, np.ctypeslib.as_ctypes(arr))
    callback_np(w, h, arr)


if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q067323177]> "e:\Work\Dev\VEnvs\py_pc064_03.08.07_test0\Scripts\python.exe" code00.py
Python 3.8.7 (tags/v3.8.7:6503f05, Dec 21 2020, 17:59:51) [MSC v.1928 64 bit (AMD64)] 64bit on win32

Array argument:  [0 1 2 3 4 5] <class 'numpy.ndarray'> (6,) uint16 1

callback_ct:
[[0 1]
 [2 3]
 [4 5]]

callback_np:
[[0 1]
 [2 3]
 [4 5]]

Done.
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.