Native Rendering Plugin with Oculus Rift

Question

I'm working on a project that offloads some rendering to a native plugin I wrote for Unity, in order to make use of instancing and other advanced graphics features. I'm developing it for a cross-platform release, but I work with a Mac so testing is done primarily with OpenGL. At this point, the plugin only renders a quad to the center of the screen colored with hex values. The plugin works as expected in a blank Unity project, but as soon as I incorporate it into my Oculus project, it begins behaving erratically.

In the Rift, the plugin's geometry draws twice, one time stretching across both eyes and another time drawing only within the bounds of the right eye. Additionally, any primitive colors I apply to the geometry are lost and the geometry appears to pick up surrounding colors; on a black screen with red text, the geometry will be mostly black with some red bleeding into the lines. As soon as my green terrain is loaded, the geometry drawn by the plugin becomes green.

Below is a screenshot of the geometry being drawn in a blank Unity project with nothing else:

And here is a screenshot of the same geometry being drawn on top of my Oculus Rift application:

Here's the creation of the vertices that I'm rendering (three coordinates and color):

Vertex verts[4] =
{
    { -0.5f,  0.5f,  0, 0xFF0000ff },
    {  0.5f,  0.5f,  0, 0xFFff0000 },
    {  0.5f, -0.5f,  0, 0xFF00ff00 },
    { -0.5f, -0.5f,  0, 0xFFff0000 },
};

Here's the draw function, called every frame within the plugin:

// OpenGL case
if (g_DeviceType == kGfxRendererOpenGL)
{
    //initialize model view matrices
    glMatrixMode (GL_MODELVIEW);
    float modelMatrix[16] =
    {
        1,0,0,0,
        0,1,0,0,
        0,0,1,0,
        0,0,0,1,
    };
    glLoadMatrixf (modelMatrix); //assign our matrix to the current MatrixMode

    //initialize projection matrix
    glMatrixMode (GL_PROJECTION);
    projectionMatrix[10] = 2.0f; //tweak projection matrix to match D3D
    projectionMatrix[14] = -1.0f;
    glLoadMatrixf (projectionMatrix);

    // Vertex layout
    glVertexPointer (3, GL_FLOAT, sizeof(verts[0]), &verts[0].x);
    glEnableClientState (GL_VERTEX_ARRAY);
    glColorPointer (4, GL_UNSIGNED_BYTE, sizeof(verts[0]), &verts[0].color);
    glEnableClientState (GL_COLOR_ARRAY);

    glDrawArrays(GL_LINE_LOOP, 0, 4);

}

Any insight from experienced native plugin/Rift graphics coders would be appreciated!

Cassation

Why the vote-for close? – Kauslick

Note that you normally also have to set the viewport prior to rendering. The OVR SDK explicitly provides a function to query the destination viewport in the render target. I'm currently not at a computer with access to my password store with the Oculus developer login, so I can't link at the reference – Kauslick

Because I'm working in Unity, I'm not currently rolling the SDK into my written code; simply using the Oculus integration package and coding on top of it. In fact, I'm not even sure if it's possible to access the OVR SDK properties from an external C++ plugin during Unity's runtime. I'd be open to adding the Oculus SDK to my plugin as a dependency, but that seems slightly redundant, if it would work at all. – Cassation

Did you ever get this working? I'm having a similar issue. – Worden

No, I never got around the issue, although I haven't tried running this code with Unity's built-in VR integration since it came out. I think it'd be a valid thing to try considering the addition of a compositor to the new VR pipeline, so I'll post here if it yields positive results when I try it. – Cassation

Answer 1

You can make use of Unity UI system to render sprites according to your needs. Here is an article from Oculus describing how to tweak Unity UI system for VR : https://developer.oculus.com/blog/unitys-ui-system-in-vr/

Outside Unity, you would use quad layers to render on top of eyes FOV. Here is layers described in Oculus Rift documentation : https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-render/#dg_render_layers

Quad Layer :

A monoscopic image that is displayed as a rectangle at a given pose and size in the virtual world. This is useful for heads-up-displays, text information, object labels and so on. By default the pose is specified relative to the user's real-world space and the quad will remain fixed in space rather than moving with the user's head or body motion. For head-locked quads, use the ovrLayerFlag_HeadLocked flag as described below.

Turk