Calling mxDestroyArray on mxArray objects returned from Matlab Compiler Runtime

Question

We've been interfacing with a library created from the Matlab Compiler. Our problem is related to an array returned from the library.

Once we're finished with the array, we'd like to free the memory, however, doing this causes occasional segmentation faults.

Here is the Matlab library (bugtest.m)::

function x = bugtest(y)

x = y.^2;

Here is the command we used to build it (creating libbugtest.so, and libbugtest.h)::

mcc -v -W lib:libbugtest -T link:lib bugtest.m

Here is our C test program (bug_destroyarray.c)::

#include <stdio.h>
#include <stdlib.h>

#include "mclmcrrt.h"
#include "libbugtest.h"

#define TESTS 15000

int main(int argc, char **argv) 
{
    const char *opts[] = {"-nojvm", "-singleCompThread"};
    mclInitializeApplication(opts, 2);  
    libbugtestInitialize();

    mxArray *output;
    mxArray *input;
    double *data;
    bool result;
    int count;

    for (count = 0; count < TESTS; count++) {

        input = mxCreateDoubleMatrix(4, 1, mxREAL);
        data = mxGetPr(input); data[0] = 0.5; data[1] = 0.2; data[2] = 0.2; data[3] = 0.1;

        output = NULL;
        result = mlfBugtest(1, &output, input);
        if (result) {
            /* HERE IS THE PROBLEMATIC LINE */
            /*mxDestroyArray(output);*/
        }

        mxDestroyArray(input);
    }

    libbugtestTerminate();
    mclTerminateApplication();
}

Here is how we compile the C program (creating bug_destroyarray)::

mbuild -v bug_destroyarray.c libbugtest.so

We believe that mxDestroyArray(output) is problematic.

We run the following to test crashing:

• On each of the 32 cluster nodes.
• Run bug_destroyarray.
• Monitor output for segmentation faults.

Roughly 10% of the time there is a crash. If this is independent across nodes then you might suppose it is crashing roughly 0.3% of the time.

When we take out that problematic line we are unable to cause it to crash.

However memory usage gradually increases when this line is not included.

From the research we've done, it seems we are not supposed to destroy the array returned, if not, how do we stop from leaking memory?

Thanks.

Nanny

Are you sure you have the signature of mlfBugtest correct? The documentation seems to indicate mlf-functions return void, not bool. – Yacht

Answer 1

Okay, I know this is a little old now, but in case it helps clarify things for anyone passing by ...

Amro provides the most pertinent information, but to expand upon it, IF you don't call the mxDestroyArray function as things stand, then you WILL leak memory, because you've set output to NULL and so the mlf function won't try to call mxDestroyArray. The corollary of this is that if you've called mxDestroyArray AND then try to call the mlf function AND output is NOT NULL, then the mlf function WILL try to call mxDestroyArray on output. The question then is to what does output point? It's a bit of a dark corner what happens to output after passing it to mxDestroyArray. I'd say it's an unwarranted assumption that it's set to NULL; it's certainly not documented that mxDestroyArray sets its argument to NULL. Therefore, I suspect what is happening is that in between your call to mxDestroyArray and the code re-executing the mlf function, something else has been allocated the memory pointed to by output and so your mlf function tries to free memory belonging to something else. Voila, seg fault. And of course this will only happen if that memory has been reallocated. Sometimes you'll get lucky, sometimes not.

The golden rule is if you're calling mxDestroyArray yourself for something that is going to be re-used, set the pointer to NULL immediately afterwards. You only really need to destroy stuff at the end of your function anyway, because you can safely re-use output variables in mlf calls.

Guy

Eamon

I wish I could find a good reference for this. The best I can find is Integrate C Shared Libraries – Montano

Answer 2

A few notes:

• I don't see singleCompThread in the list of allowed options for mclInitializeApplication.

• The recommended way to compile your C program is to dynamically link against the compiled library:

  mbuild -v -I. bug_destroyarray.c -L. -lbugtest
  

• At the top of your C program, just include the generated header file, it will include other headers in turn. From looking at the generated header, it has:

  #pragma implementation "mclmcrrt.h"
  #include "mclmcrrt.h"
  

  I dont know the exact meaning of this pragma line, but maybe it matters with GCC compilers..

• The fact that both mlx/mlf generated functions return booleans is undocumented. But looking at the header files, both signatures do indeed return a bool:

  extern bool mlxBugtest(int nlhs, mxArray *plhs[], int nrhs, mxArray *prhs[]);
  extern bool mlfBugtest(int nargout, mxArray** x, mxArray* y);
  

I tried your code and it works just fine with no segfaults. As I dont have access to a cluster of computers, my testing was only done on my local machine (WinXP with R2013a).

I had to remove both MCR initialization options for it to work (specifically the nojvm caused a runtime error). Below is the full code with slight modifications. It took around 10 seconds to run:

#include <stdio.h>
#include <stdlib.h>
#include "libbugtest.h"

#define TESTS 15000

int main() 
{
    mxArray *output, *input;
    double *data;
    int count;
    bool result;

    if( !mclInitializeApplication(NULL,0) ) {
        fprintf(stderr, "Could not initialize the application.\n");
        return EXIT_FAILURE;
    }
    if ( !libbugtestInitialize() ) {
        fprintf(stderr, "Could not initialize the library.\n");
        return EXIT_FAILURE;
    }

    for (count = 0; count < TESTS; count++) {
        input = mxCreateDoubleMatrix(4, 1, mxREAL);
        data = mxGetPr(input);
        data[0] = 0.5; data[1] = 0.2; data[2] = 0.2; data[3] = 0.1;

        output = NULL;
        result = mlfBugtest(1, &output, input);
        if (!result) {
            fprintf(stderr, "call failed on count=%d\n", count);
            return EXIT_FAILURE;
        }

        mxDestroyArray(output); output = NULL;
        mxDestroyArray(input); input = NULL;
    }

    libbugtestTerminate();
    mclTerminateApplication();

    return EXIT_SUCCESS;
}

Also the compilation step is a bit different on Windows, since we statically link against the import lib (which inserts a stub to dynamically load the DLL on runtime):

mbuild -v -I. bug_destroyarray.c libbugtest.lib

Connecticut

Answer 3

Thanks for the detailed reply Amro.

We tried changing our compilation steps to the recommended ones, with no success.

The following fixed our seg-faulting problem:

• Do not set output = NULL at each iteration, instead do it once outside of the loop.
• Do not call mxDestroyArray(output) inside the loop, reference: here.

Our misunderstanding was that (it seems) you are supposed to reuse mxArray pointers which you pass to MATLAB functions. It makes things slightly cumbersome on our side as we need to be careful reusing this pointer.

However, memory is completely stable, and we've not had a crash since.

Nanny

I think you confusing things on that second point. The page you linked to is intended in the context of MEX-functions (where you do not want to free the left hand side returned to MATLAB). As for the other point, the docs clearly say that "If the output variables you pass in to an mlf function are not NULL, the mlf function will attempt to free them using mxDestroyArray.". So there should be any difference whether you explicitly free the memory yourself or let the mlf function do it – Connecticut

Ahh ok, thanks for the explanation. I shouldn't have said ever, just meant not inside the loop. It all makes sense now, and is working correctly. Cheers. – Nanny