Python ctypes definition for c struct

Question

I am trying to call some c code generated by the Matlab coder. Matlab uses a c struct called emxArray to represent matrices (documented here: http://www.mathworks.co.uk/help/fixedpoint/ug/c-code-interface-for-unbounded-arrays-and-structure-fields.html).

struct emxArray_real_T
{
    double *data;
    int *size;
    int allocatedSize;
    int numDimensions;
    boolean_T canFreeData;
};

I have little experience ctypes and am struggling to create an equivalent struct that I can then use to pass vectors back and forth to the functions defined in the c .so

Here is where I have got so far in python...

class EmxArray(ctypes.Structure):
    """ creates a struct to match emxArray_real_T """

    _fields_ = [('data', ctypes.POINTER(ctypes.c_double)),
                ('size', ctypes.POINTER(ctypes.c_int)),
                ('allocatedSize', ctypes.c_int),
                ('numDimensions', ctypes.c_int),
                ('canFreeData', ctypes.c_bool)]    

However if I define this:

data = (1.1, 1.2, 1.3, 1.4)
L = len(data)

x = EmxArray()
x.data = (ctypes.c_double * L)(*data)
x.data = (ctypes.c_int * 1)(L)    

this then works

print len(x.data[:L]) 

for v in x.data[:L]: print v

Edit: I have tidied up and adopted Roland's suggestion and can extract the data using

data_out = x.data[:L]

I need to investigate further to see if I can successfully use this struct to pass and receive data from the c code.

Solution

Implementing the ctypes struct as suggested by Roland didn't work - the returned values were garbage, I never worked out why as I pursued a python based implementation of lilbil's answer. I've accepted that answer as it was closest...

I'll document my solution here as it might save someone else wasting as much time as I have.

Firstly I've generated a simple matlab function that multiplies each element of a function by itself & used the coder to compile this to a c .so. This is imported to python using ctypes. The code is as follows...

import ctypes

LIBTEST = '..../dll/emx_test/'
EMX = ctypes.cdll.LoadLibrary(LIBTEST + 'emx_test.so')
init = EMX.emx_test_initialize()

# Create a data structure to hold the pointer generated by emxCreateWrapper...
class Opaque(ctypes.Structure):
    pass

# make some random data to pass in
data_in = [1., 2., 4., 8., 16.]
L = len(data_in)
# create an empty array of the same size for the output
data_ou = [0] * L

# put this in a ctypes array
ina = (ctypes.c_double * L)(*data_in)
oua = (ctypes.c_double * L)(*data_ou)
# create a pointer for these arrays & set the rows and columns of the matrix
inp = ctypes.pointer(ina)
oup = ctypes.pointer(oua)

nrows = ctypes.c_int(1)
ncols = ctypes.c_int(L)

# use EMX.emxCreateWrapper_real_T(double *data, int rows, int cols) to generate an emx wrapping the data 
# input arg types are a pointer to the data NOTE its not great to have to resize the ctypes.c_double but cant see another way
EMX.emxCreateWrapper_real_T.argtypes = [ctypes.POINTER(ctypes.c_double * L), ctypes.c_int, ctypes.c_int]
# a pointer to the emxArray is returned and stored in Opaque
EMX.emxCreateWrapper_real_T.restype = ctypes.POINTER(Opaque)
# use emxCreateWrapper
in_emx = EMX.emxCreateWrapper_real_T(inp, nrows, ncols)
ou_emx = EMX.emxCreateWrapper_real_T(oup, nrows, ncols)

# so now we have to emx's created and have pointers to them we can run the emx_test
# emx test looks like this in matlab
#
# function res = emx_test ( in )
#     res = in .* in;
# end
#
# so basically it multiplies each element of the matrix by itself
# 
# therefore [1., 2., 4., 8., 16.] should become [1., 4., 8., 64., 256.]

EMX.emx_test(in_emx, ou_emx)

# and voila...that's what we get
print 'In: ', ina[:L]
print 'Out:', oua[:L]

Output:

In: [1.0, 2.0, 4.0, 8.0, 16.0]
Out:[1.0, 4.0, 16.0, 64.0, 256.0]

Thank you to everyone for your time & suggestions.

Brownlee

I feel like you probably don't need the sz and ...*sz))' parts of the EmxArray` class. – Taker

The reason I did that was that data will be an array it would be best if I could allow for a variable sized array however. – Brownlee

Answer 1

I'm not well-versed with the Python-C interface, so what I'm suggesting may be less than ideal. My guess is that likely the crash is because x->data is never initialized and the memory to which it is pointing isn't allocated.

An approach I have taken when interfacing to MATLAB Coder generated code from other languages in the presence of emxArray arguments is to hand-write a C interface function that provides a simpler API. This relieves the burden of needing to construct an emxArray in the other environment (Android Java in my particular case). If the generated function foo takes and returns a 2-D double array, then something like the following could work:

void foo(double *x, int *szx, double **y, int *szy);

This function would take a pointer to the input data and its size and provide a pointer to the output data and its size. The implementation would look something like:

void foo(double *x, int *szx, double **y, int *szy) 
{
  emxArray_real_T *pEmx;
  emxArray_real_T *pEmy;

  /* Create input emxArray assuming 2-dimensional input */
  pEmx = emxCreateWrapper_real_T(x, szx[0], szx[1]);

  /* Create output emxArray (assumes that the output is not */
  /* written before allocation occurs) assuming 2-D output  */
  pEmy = emxCreateWrapper_real_T(NULL, 0, 0);

  /* Call generated code (call foobar_initialize/terminate elsewhere) */
  foobar(pEmx, pEmy);

  /* Unpack result - You may want to MALLOC storage in *y and */
  /* MEMCPY there alternatively                               */
  *y = pEmy->data;
  szy[0] = pEmy->size[0];
  szy[1] = pEmy->size[1];

  /* Clean up any memory allocated in the emxArrays (e.g. the size vectors) */
  emxDestroyArray_real_T(pEmx);
  emxDestroyArray_real_T(pEmy);
}

You should be able to call this function from Python more simply and pass in the desired data as needed.

My other answer has more details on the emxArray_* functions found in the file foobar_emxAPI.h.

Veta

Thanks for this comprehensive answer. I think this is probably the way to go so I'll explore this route today. I'm currently working on a dummy matlab script that just takes an emx_array squares it and returns it as an emx_array. The library I eventually need to work with defines multiple functions & therefore wrapping is more cumbersome, I wonder if it would be possible to define a general purpose function that takes a vector and defines an emx_array that can be passed to a function and visa versa... – Brownlee

That could be possible. How do the types and complexities of the arguments to the library differ? Generally the differences can exist in type, complexity or number of dimensions of the arguments. A change in those would require a behavioral change in the conversion routine. Can your conversion routine be written in C++? If so, it could be templatized with specializations for different types that know how to call the proper functions in foo_emxApi.h. – Veta

An alternative thought I had is that you may be able to declare a pointer to an appropriate emxArray type in Python. Then you could call into the generated emxCreate* or emxCreateWrapper* functions to initialize that pointer from Python code. This should minimize the amount of C you need to write and these functions have simple signatures with numeric arguments. This may also let you write the vector <--> emxArray conversion function in Python. – Veta

Answer 2

Just create a pointer, assign the data afterwards;

import ctypes

class EmxArray(ctypes.Structure):
    """ creates a struct to match emxArray_real_T """

    _fields_ = [('data', ctypes.POINTER(ctypes.c_double)),
                ('size', ctypes.POINTER(ctypes.c_int)),
                ('allocatedSize', ctypes.c_int),
                ('numDimensions', ctypes.c_int),
                ('canFreeData', ctypes.c_bool)]

data = (1.3, 3.5, 2.7, 4.1)
L = len(data)

e = EmxArray()
e.data = (ctypes.c_double * L)(*data)
e.size = (ctypes.c_int * 1)(L)
# et cetera

Audacity