Using OpenGL in Matlab to get depth buffer
Asked Answered
C

2

10

Ive asked a similar question before and didnt manage to find a direct answer.

Could someone provide sample code for extracting the depth buffer of the rendering of an object into a figure in Matlab?

So lets say I load an obj file or even just a simple surf call, render it and now want to get to its depth buffer then what code will do that for me using both Matlab and OpenGL. I.e. how do I set this up and then access the actual data?

I essentially want to be able to use Matlabs powerful plotting functions and then be able to access the underlying graphics context for getting the depth buffer out.

NOTE: The bounty specifies JOGL but that is not a must. Any code which acts as above and can provide me with the depth buffer after running it in Matlab is sufficient)

Chick answered 14/1, 2012 at 13:20 Comment(4)
you could always try to offer a bounty to increase the attention a question receives.Dealer
I intend to but only discovered when I posted the question that I need to wait two days until this current question becomes eligible.Chick
Do you accept answers that involve Matlab only? I have no idea about JOGL.Oldenburg
Yes, the question is specifically Matlab related. We have a number of things we could do if we could usefully access the underlying OpenGL layer of a figure in Matlab so Matlab is crucial.Chick
G
14

depthmap thing

Today, I went drinking with my colleagues, and after five beers and some tequillas I found this question and thought, "have at ya!" So I was struggling for a while but then I found a simple solution using MEX. I theorized that the OpenGL context, created by the last window, could be left active and therefore could be accessible from "C", if the script ran in the same thread.

I created a simple "C" program which calls one matlab function, called "testofmyfilter" which plots frequency response of a filter (that was the only script I had at hand). This is rendered using OpenGL. Then the program uses glGetViewport() and glReadPixels() to get to the OpenGL buffers. Then it creates a matrix, fills it with the depth values, and passes it to the second function, called "trytodisplaydepthmap". It just displays the depthmap using the imshow function. Note that the MEX function is allowed to return values as well, so maybe the postprocessing would not have to be another function, but I'm in no state to be able to understand how it's done. Should be trivial, though. I'm working with MEX for the first time today.

Without further delay, there are source codes I used:

testofmyfilter.m

imp = zeros(10000,1);
imp(5000) = 1;
% impulse

[bwb,bwa] = butter(3, 0.1, 'high');
b = filter(bwb, bwa, imp);
% filter impulse by the filter

fs = 44100; % sampling frequency (all frequencies are relative to fs)
frequency_response=fft(b); % calculate response (complex numbers)
amplitude_response=20*log10(abs(frequency_response)); % calculate module of the response, convert to dB
frequency_axis=(0:length(b)-1)*fs/length(b); % generate frequency values for each response value
min_f=2;
max_f=fix(length(b)/2)+1; % min, max frequency

figure(1);
lighting gouraud
set(gcf,'Renderer','OpenGL')

semilogx(frequency_axis(min_f:max_f),amplitude_response(min_f:max_f),'r-') % plot with logarithmic axis using red line
axis([frequency_axis(min_f) frequency_axis(max_f) -90 10])  % set axis limits

xlabel('frequency [Hz]');
ylabel('amplitude [dB]'); % legend

grid on % draw grid

test.c

//You can include any C libraries that you normally use
#include "windows.h"
#include "stdio.h"
#include "math.h"
#include "mex.h"   //--This one is required

extern WINAPI void glGetIntegerv(int n_enum, int *p_value);

extern WINAPI void glReadPixels(int     x, 
    int     y, 
    int     width, 
    int     height, 
    int     format, 
    int     type, 
    void *      data);

#define GL_VIEWPORT                       0x0BA2
#define GL_DEPTH_COMPONENT                0x1902
#define GL_FLOAT                          0x1406

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    int viewport[4], i, x, y;
    int colLen;
    float *data;
    double *matrix;
    mxArray *arg[1];

    mexCallMATLAB(0, NULL, 0, NULL, "testofmyfilter");
    // call an .m file which creates OpenGL window and draws a plot inside

    glGetIntegerv(GL_VIEWPORT, viewport);
    printf("GL_VIEWPORT = [%d, %d, %d, %d]\n", viewport[0], viewport[1], viewport[2], viewport[3]);
    // print viewport dimensions, should be [0, 0, m, n]
    // where m and n are size of the GL window

    data = (float*)malloc(viewport[2] * viewport[3] * sizeof(float));
    glReadPixels(0, 0, viewport[2], viewport[3], GL_DEPTH_COMPONENT, GL_FLOAT, data);
    // alloc data and read the depth buffer

    /*for(i = 0; i < 10; ++ i)
        printf("%f\n", data[i]);*/
    // debug

    arg[0] = mxCreateNumericMatrix(viewport[3], viewport[2], mxDOUBLE_CLASS, mxREAL);
    matrix = mxGetPr(arg[0]);
    colLen = mxGetM(arg[0]);
    printf("0x%08x 0x%08x 0x%08x %d\n", data, arg[0], matrix, colLen); // debug
    for(x = 0; x < viewport[2]; ++ x) {
        for(y = 0; y < viewport[3]; ++ y)
            matrix[x * colLen + y] = data[x + (viewport[3] - 1 - y) * viewport[2]];
    }
    // create matrix, copy data (this is stupid, but matlab switches
    // rows/cols, also convert float to double - but OpenGL could have done that)

    free(data);
    // don't need this anymore

    mexCallMATLAB(0, NULL, 1, arg, "trytodisplaydepthmap");
    // pass the array to a function (returnig something from here
    // is beyond my understanding of mex, but should be doable)

    mxDestroyArray(arg[0]);
    // cleanup

    return;
}

trytodisplaydepthmap.m:

function [] = trytodisplaydepthmap(depthMap)

figure(2);
imshow(depthMap, []);
% see what's inside

Save all of these to the same directory, compile test.c with (type that to Matlab console):

mex test.c Q:\MATLAB\R2008a\sys\lcc\lib\opengl32.lib

Where "Q:\MATLAB\R2008a\sys\lcc\lib\opengl32.lib" is path to "opengl32.lib" file.

And finally execute it all by merely typing "test" in matlab console. It should bring up a window with filter frequency response, and another window with the depth buffer. Note the front and back buffers are swapped at the moment "C" code reads the depth buffer, so it might be required to run the script twice to get any results (so the front buffer which now contains the results swaps with back buffer again, and the depth can be read out). This could be done automatically by "C", or you can try including getframe(gcf); at the end of your script (that reads back from OpenGL as well so it swaps the buffers for you, or something).

This works for me in Matlab 7.6.0.324 (R2008a). The script runs and spits out the following:

>>test
GL_VIEWPORT = [0, 0, 560, 419]
0x11150020 0x0bd39620 0x12b20030 419

And of course it displays the images. Note the depth buffer range depends on Matlab, and can be quite high, so making any sense of the generated images may not be straightforward.

Glue answered 18/1, 2012 at 18:27 Comment(7)
oh, i missed the requirements. of course this works with surf() too. just tried the code at: mathworks.com/help/techdoc/ref/surf.html and got: luki.webzdarma.cz/up/depthmap.png the obviousGlue
You deserve a medal, another beer and the 300 rep.Chick
Can I recommend you put all that together and make it into a nice mathworks submission? Im not the only one who finds this useful.Chick
And for anyone else who wants to test this out I recommend swapping the entire contents of testofmyfilter.m to be simply "peaks"Chick
Hello, thanks for all the praise. I'm not much into matlab, I mostly do my math in mspaint or in "C". So ... if you think someone may find this useful, you are welcome to make the submission yourself. I don't require any credit.Glue
Unfortunately, any GL calls (e.g. glGetIntegerv) in a mex function seems to crash matlab 2014b on Mac OS X. Any solution for mac?Coggins
@mangledorf That is quite strange, although I remember I only tried this on Windows. Can you try to debug and post more details on the crash, perhaps in a new question?Glue
C
2

the swine's answer is the correct one. Here is a slightly formatted and simpler version that is cross-platform.

Create a file called mexGetDepth.c

#include "mex.h"   

#define GL_VIEWPORT                       0x0BA2
#define GL_DEPTH_COMPONENT                0x1902
#define GL_FLOAT                          0x1406

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    int viewport[4], i, x, y;
    int colLen;
    float *data;
    double *matrix;

    glGetIntegerv(GL_VIEWPORT, viewport);
    data = (float*)malloc(viewport[2] * viewport[3] * sizeof(float));
    glReadPixels(0, 0, viewport[2], viewport[3], GL_DEPTH_COMPONENT, GL_FLOAT, data);

    plhs[0] = mxCreateNumericMatrix(viewport[3], viewport[2], mxDOUBLE_CLASS, mxREAL);
    matrix = mxGetPr(plhs[0]);
    colLen = mxGetM(plhs[0]);

    for(x = 0; x < viewport[2]; ++ x) {
        for(y = 0; y < viewport[3]; ++ y)
            matrix[x * colLen + y] = data[x + (viewport[3] - 1 - y) * viewport[2]];
    }

    free(data);
    return;
}

Then if youre on windows compile using

mex mexGetDepth.c "path to OpenGL32.lib"

or if youre on a nix system

mex mexGetDepth.c "path to opengl32.a"

Then run the following small script to test out the new function

peaks;
figure(1);
depthData=mexGetDepth;
figure
imshow(depthData);
Chick answered 19/1, 2012 at 1:5 Comment(1)
do we need to declare the glGetIntegerv and glReadPixels fucntions somewhere? In the original code, it's done using extern WINAPI void..., but I'm not sure how we would do it in linuxVasodilator

© 2022 - 2024 — McMap. All rights reserved.