Passing a NumPy 3d array to a C function with a triple pointer as an argument

Question

I am experimenting with the use of the ctypes package in Python. Currently attempting to pass a 3d NumPy array to a C function that takes a triple pointer as an argument and updates the values of the elements of the array. The C function is:

void effect_array(int ***ptr, int rows, int cols, int depth)
{
    for(int i = 0; i < rows; i++)
    {
        for(int j = 0; j < cols; j++)
        {
            for(int k = 0; k < depth; k++)
            {
                ptr[i][j][k] *= 2;
            }
        }
    }
}

Currently, I have tried:

import ctypes as ct
import numpy as np

arr = np.array([
    [
        [1, 2, 3, 4],
        [5, 6, 7, 8],
        [9, 10, 11, 12]
    ],
    [
        [13, 14, 15, 16],
        [17, 18, 19, 20],
        [21, 22, 23, 24]
    ]
])

arr_c = np.ascontiguousarray(arr)

lib = ct.CDLL("path/to/share_object.so")
_effect_array = lib.effect_array
_effect_array.argtypes = [ct.POINTER(ct.POINTER(ct.POINTER(ct.c_int))), ct.c_int, ct.c_int, ct.c_int]
_effect_array.restype = None

rows, cols, depth = arr.shape
t_ptr = (ct.POINTER(ct.POINTER(ct.c_int)) * rows)()
for i in range(rows):
    t_ptr[i] = (ct.POINTER(ct.c_int) * cols)()
    for j in range(cols):
        t_ptr[i][j] = arr_c[i][j].ctypes.data_as(ct.POINTER(ct.c_int))
        
print("Original array =")
print(arr_c)
print()
_effect_array(t_ptr, rows, cols, depth)
print("Array after pass to function =")
print(arr_c)

This has resulted in an output of:

Original array =
[[[ 1  2  3  4]
  [ 5  6  7  8]
  [ 9 10 11 12]]

 [[13 14 15 16]
  [17 18 19 20]
  [21 22 23 24]]]

Array after pass to function =
[[[ 2  4  3  4]
  [10 12  7  8]
  [18 20 11 12]]

 [[26 28 15 16]
  [34 36 19 20]
  [42 44 23 24]]]

What I would like to happen would be:

Original array =
[[[ 1  2  3  4]
  [ 5  6  7  8]
  [ 9 10 11 12]]

 [[13 14 15 16]
  [17 18 19 20]
  [21 22 23 24]]]

Array after pass to function =
[[[ 2  4  6  8]
  [10 12  14  16]
  [18 20 22 24]]

 [[26 28 30 32]
  [34 36 38 40]
  [42 44 46 48]]]

I am not sure why the C function is not able to access the elements beyond the first two in a row. My understanding is that my current attempt is provided access to the start of the array and that it should be able to iterate through the rest of the array as can be done in this C example:

#include <stdio.h>
#include <stdlib.h>

void effect_array(int ***ptr, int rows, int cols, int depth)
{
    for(int i = 0; i < rows; i++)
    {
        for(int j = 0; j < cols; j++)
        {
            for(int k = 0; k < depth; k++)
            {
                ptr[i][j][k] *= 2;
            }
        }
    }
}

int main()
{
    int arr[2][2][2] = {
        {
            {1,2},
            {3,4}
        },
        {
            {5,6},
            {7,8}
        }
    };
    
    int arr2[5];
    int *p = arr2;
    
    int ***ptr = (int ***)malloc(2 * sizeof(int **));
    for(int i = 0; i < 2; i++)
    {
        ptr[i] = (int **)malloc(2 * sizeof(int *));
        for(int j = 0; j < 2; j++)
        {
            ptr[i][j] = &arr[i][j][0];
        }
    }
    
    printf("Print array before:\n");
    for(int i = 0; i < 2; i++)
    {
        for(int j = 0; j < 2; j++)
        {
            for(int k = 0; k < 2; k++)
            {
                printf("%d ", arr[i][j][k]);
            }
            printf("\n");
        }
        printf("\n");
    }
    
    effect_array(ptr, 2, 2, 2);
    
    printf("Print array after:\n");
    for(int i = 0; i < 2; i++)
    {
        for(int j = 0; j < 2; j++)
        {
            for(int k = 0; k < 2; k++)
            {
                printf("%d ", arr[i][j][k]);
            }
            printf("\n");
        }
        printf("\n");
    }
    
    for(int i = 0; i < 2; i++) free(ptr[i]);
    free(ptr);

    return 0;
}

This is also based off the discussion on a previous question I asked, credit to @SR143 for the working C example.

Any and all help is greatly appreciated.

This kinda just looks like you might have compiled the wrong C code, with a typo in it. — user2357112
– user2357112, Commented Feb 2 at 15:50
@user2357112 the C snippet I provided is the one I am using, do you see a typo in there? — frank
– frank, Commented Feb 2 at 15:59
Another potential issue: I think you have to keep Python references to those ctypes arrays you create, or they'll get reclaimed, and you can end up accessing freed memory. t_ptr[i] = (ct.POINTER(ct.c_int) * cols)() and t_ptr[i][j] = arr_c[i][j].ctypes.data_as(ct.POINTER(ct.c_int)) both discard the only remaining reference to the new array. — user2357112
– user2357112, Commented Feb 2 at 16:07
@user2357112 I apologize for asking for the hand holding, but is there a more appropriate way to hold on to those? Should they be made into Python objects and then used as rvalues? — frank
– frank, Commented Feb 2 at 16:15
I'd just stick 'em all in a list and keep the list around until I'm done with the memory. — user2357112
– user2357112, Commented Feb 2 at 16:17

Mark Tolonen · Accepted Answer · 2025-02-02 19:02:09Z

2

In your original Python code, the dtype of arr is np.int64. Add dtype=np.int32 to your np.array declaration and the code works.

However, be aware that you can pass a multidimensional numpy array directly to C code instead by using an int* and pointer math on the contiguous C array and be more efficient since it will save you the trouble of creating the pointer array. An np.ctypeslib.ndpointer can be used with .argtypes for better type-checking as well.

Example:

test.c (Windows example):

__declspec(dllexport)
void affect_array(int *ptr, int rows, int cols, int depth) {
    for(int i = 0; i < rows; i++) {
        for(int j = 0; j < cols; j++) {
            for(int k = 0; k < depth; k++) {
                ptr[i*cols*depth + j*depth + k] *= 2;
            }
        }
    }
}

test.py

import ctypes as ct
import numpy as np

arr = np.array([[[1, 2, 3, 4],
                 [5, 6, 7, 8],
                 [9, 10, 11, 12]],
                [[13, 14, 15, 16],
                 [17, 18, 19, 20],
                 [21, 22, 23, 24]]], dtype=np.int32)

lib = ct.CDLL('./test')
affect_array = lib.affect_array
# Require 3D numpy array of int32 as first parameter.  It will be type-checked.
affect_array.argtypes = np.ctypeslib.ndpointer(dtype=np.int32, ndim=3), ct.c_int, ct.c_int, ct.c_int
affect_array.restype = None

print('Original array =')
print(arr)
print()
affect_array(arr, *arr.shape)  # * unpacks tuple as 3 additional parameters.
print('Array after pass to function =')
print(arr)

Output:

Original array =
[[[ 1  2  3  4]
  [ 5  6  7  8]
  [ 9 10 11 12]]

 [[13 14 15 16]
  [17 18 19 20]
  [21 22 23 24]]]

Array after pass to function =
[[[ 2  4  6  8]
  [10 12 14 16]
  [18 20 22 24]]

 [[26 28 30 32]
  [34 36 38 40]
  [42 44 46 48]]]

edited Feb 2 at 19:02

answered Feb 2 at 18:39

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

frank Feb 2 at 19:19

Thank you so much! Am I correct in assuming that the reason the original method required specifying the data type for the array is that the addresses of the elements of the array would be off due to the difference in size for numpy.int32 (32-bit) and numpy.int64 (64-bit)?

Mark Tolonen Feb 2 at 19:22

@Frank Yes, the memory size of the elements in C and Python needs to match. Adjacent elements were expected to be four bytes apart in the C code, not eight.

Collectives™ on Stack Overflow

Passing a NumPy 3d array to a C function with a triple pointer as an argument

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related