OpenGL - object outline

Question

I'm trying to implement a selection-outline feature. This is what I get up to now.

As you can see, the objects are selected correctly when the mouse hovers and a contour is drawn around the selected object.

What I would like to do now is to outline the visible edges of the object in this way

In the image on the left is what I have now, and in the right image is what I want to achieve.

This is the procedure I use now.

void paintGL()
{
    /* ... */

    int w = geometry().width();
    int h = geometry().height();

    glEnable(GL_DEPTH_TEST);
    glDepthFunc(GL_LESS);

    glEnable(GL_STENCIL_TEST);
    glStencilFunc(GL_NOTEQUAL, 1, 0xFF);
    glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
    glStencilMask(0xFF);

    setClearColor(Qt::GlobalColor::darkGray);

    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

    glStencilMask(0x00);
    DrawGeometry();

    if (HoveredSphere != RgbFromColorToString(Qt::GlobalColor::black))
    {
        glBindFramebuffer(GL_FRAMEBUFFER, addFBO(FBOIndex::OUTLINE));
        {
            glStencilFunc(GL_ALWAYS, 1, 0xFF);
            glStencilMask(0xFF);

            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

            DrawOutline(HoveredSphere, 1.0f - 0.025f);
        }
        glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebufferObject());

        glBindFramebuffer(GL_READ_FRAMEBUFFER, addFBO(FBOIndex::OUTLINE));
        glBindFramebuffer(GL_DRAW_FRAMEBUFFER, defaultFramebufferObject());
        {
            // copy stencil buffer
            GLbitfield mask = GL_STENCIL_BUFFER_BIT;
            glBlitFramebuffer(0, 0, w, h, 0, 0, w, h, mask, GL_NEAREST);

            glStencilFunc(GL_NOTEQUAL, 1, 0xFF);
            glStencilMask(0x00);

            glDepthFunc(GL_LEQUAL);
            DrawOutline(HoveredSphere, 1.0f);
        }
        glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebufferObject());
    }

    update();
}

Where DrawGeometry draws all the objects, and DrawOutline draws the selected object scaled by the factor passed as the second parameter.

Thanks for any suggestions.

Aerodynamics

Two ways I can think of: One way would be to compute the "ring" where your shapes overlap beforehand and then draw that ring with GL_LINES or similiar. Another way would be to render the image first to a framebuffer where the highlighted shape gets a special value and then postprocess it with a Sobel operator. – Mastigophoran

How about this Silhouette-Outlined shader – Nadya

@MichaelMahn Thank you! I like the rings-idea, I want to try to implement it but I have some doubt. 1st performance. The final scene has 1000s spheres, for each I should calculate every possible intersection. This is something that I can do 1 time offline when loading the model. 2nd ring approximation. GL_LINES can give only an approximation of a circle, while I depict the spheres as impostor (using ray-tracing), obtaining a "perfect" sphere. I should find a way to depict also a "perfect" ring. I like less the sobel-idea because I don't see if I could get the correct result with this approach. – Aerodynamics

@Nadya Thank you! Do you think that with your approach I can get the desired effect? Rapidly reading, it seems that your approach outlines only the outside edge of an object, as I do now, but I'm trying to achieve something different. Sorry if I don't get your solution. – Aerodynamics

@Arctic Pi Yes, the ring thing will definitely be better if you can do it offline just once. If your spheres move and you have to do it once per frame, you'll have to take performance into account. Sobel operator should also work if I don't misunderstand your problem. You could use the stencil buffer which you already have and if the fragment was rendered by the highlighted sphere and at least one of the neighbour fragments wasn't, then this fragment has to use the outlining color. – Mastigophoran

@MichaelMahn Thank you so much for your answer. You gave me an idea on how to achieve my goal and I did it. This afternoon I will show you the result in an answer to the question. There is still an improvement I need, for which I will ask you for a tip in my answer. Thanks again! – Aerodynamics

@ArcticPi You can outline only the selected area too ... It just a matter of what color you are looking for ... If you use index buffer for the ray-picking by mouse then you do not even need to add any other rendering pass ... – Nadya

Answer 1

By following the tips of @MichaelMahn, I found a solution.

First of all, I draw the silhouette of the visible parts of the selected object in a texture.

And then I use this texture to calculate the outline by checking the neighboring pixels to figure out whether or not I stand on the edge of the silhouette.

outline fragment shader

#version 450

uniform sampler2D silhouette;

in FragData
{
    smooth vec2 coords;
} frag;

out vec4 PixelColor;

void main()
{
    // if the pixel is black (we are on the silhouette)
    if (texture(silhouette, frag.coords).xyz == vec3(0.0f))
    {
        vec2 size = 1.0f / textureSize(silhouette, 0);

        for (int i = -1; i <= +1; i++)
        {
            for (int j = -1; j <= +1; j++)
            {
                if (i == 0 && j == 0)
                {
                    continue;
                }

                vec2 offset = vec2(i, j) * size;

                // and if one of the neighboring pixels is white (we are on the border)
                if (texture(silhouette, frag.coords + offset).xyz == vec3(1.0f))
                {
                    PixelColor = vec4(vec3(1.0f), 1.0f);
                    return;
                }
            }
        }
    }

    discard;
}

paintgl

void paintGL()
{
    int w = geometry().width();
    int h = geometry().height();

    setClearColor(Qt::GlobalColor::darkGray);

    glEnable(GL_DEPTH_TEST);
    glDepthFunc(GL_LESS);

    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    DrawGeometry();

    // if we hover a sphere
    if (HoveredSphere != RgbFromColorToString(Qt::GlobalColor::black))
    {
        glBindFramebuffer(GL_READ_FRAMEBUFFER, defaultFramebufferObject());
        glBindFramebuffer(GL_DRAW_FRAMEBUFFER, addFBO(FBOIndex::SILHOUETTE));
        {
            // copy depth buffer
            GLbitfield mask = GL_DEPTH_BUFFER_BIT;
            glBlitFramebuffer(0, 0, w, h, 0, 0, w, h, mask, GL_NEAREST);

            // set clear color
            setClearColor(Qt::GlobalColor::white);
            // enable depth test
            glEnable(GL_DEPTH_TEST);
            glDepthFunc(GL_LEQUAL);
            // clear color buffer
            glClear(GL_COLOR_BUFFER_BIT);

            // draw silhouette
            DrawSilhouette(HoveredSphere);
        }
        glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebufferObject());

        // clear depth buffer
        glClear(GL_DEPTH_BUFFER_BIT);

        // draw outline
        DrawOutline();
    }
}

PROBLEM :: Now I'd like to parameterize the width of the contour, whose thickness is currently fixed at 1 pixel.

Thank you so much for any suggestion!

Aerodynamics

By the way, GLSL has a function textureOffset() that would be simpler and cleaner (as textureOffset(silhouette, frag.coords, ivec2(i, j)).xyz, I think?) than what your current code does with texture(silhouette, frag.coords + vec2(i, j) * (1.0f / textureSize(silhouette, 0))).xyz. – Aquilar

Regarding parameterising the width, I guess what you'd need to do is check not just the next pixel e.g. (x + 1, y) but also the one after it (x + 2, y) and, if either is true, produce white. You could perhaps have something like: ivec2 directions[4] = ivec2[4](ivec2(-1, 0), ivec2(1, 0), ivec2(0, -1), ivec2(0, 1)); for (uint i = 0; i < 4; i++) { for (uint j = 1; j <= width; j++) { if (textureOffset(silhouette, frag.coords + directions[i] * j) != vec3(1.0f)) { break; } else { PixelColor = vec4(vec3(1.0f), 1.0f); return; } } } — but I haven't tested that mind you. – Aquilar

(P.S. that width should probably be a constant value within the shader, instead of a uniform, so the shader compiler can unroll that loop for better performance.) – Aquilar

@Andrea thanks a lot for your answer! I first tried to adopt your approach, but I obtained incorrect results. So I turned to the solution described in my answer below. I still have a small problem, if you want to take a look. ;) Thanks a lot! – Aerodynamics

@Andrea I have also tried the textureOffset method, but unfortunately it seems that Qt5 does not support it :( Thanks for the tip! ;) – Aerodynamics

Answer 2

Thanks to the advice of @Andrea I found the following solution.

outline fragment shader

#version 450

uniform sampler2D silhouette;

in FragData
{
    smooth vec2 coords;
} frag;

out vec4 PixelColor;

void main()
{
    // outline thickness
    int w = 3;

    // if the pixel is black (we are on the silhouette)
    if (texture(silhouette, frag.coords).xyz == vec3(0.0f))
    {
        vec2 size = 1.0f / textureSize(silhouette, 0);

        for (int i = -w; i <= +w; i++)
        {
            for (int j = -w; j <= +w; j++)
            {
                if (i == 0 && j == 0)
                {
                    continue;
                }

                vec2 offset = vec2(i, j) * size;

                // and if one of the pixel-neighbor is white (we are on the border)
                if (texture(silhouette, frag.coords + offset).xyz == vec3(1.0f))
                {
                    PixelColor = vec4(vec3(1.0f), 1.0f);
                    return;
                }
            }
        }
    }

    discard;
}

Now I still have a small problem when the selected object is at the edge of the window.

As you can see, the outline is cut sharply.

I tried to "play" with glTexParameter on the parameters GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T.

In the image above you can see the effect I get with

glTexParameters(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameters(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

while in the image below you can see the effect I get with

glTexParameterfv(GL_TEXTURE_2D, GL_TEXTURE_BORDER_COLOR, &(QVector4D (1.0f, 1.0f, 1.0f, 1.0f)[0]));
glTexParameters(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_BORDER);
glTexParameters(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_BORDER);

I would like the outline to show up at the edge of the window, but only where necessary.

Thanks a lot!

Aerodynamics

Hmm. You could maybe use a different check for the window border: if(texture(silhouette, frag.coords) != vec3(1.0) && (gl_FragCoord.x < 2 || gl_FragCoord.y < 2 || gl_FragCoord.x >= viewportWidth - 2 || gl_FragCoord.y >= viewportHeight - 2)). Alternatively, you can adjust the texture repeat settings for the texture to handle what happens when reading pixels over the edge (I'm assuming it's currently CLAMP_TO_EDGE) so it will see white pixels and draw the outline. – Aquilar

Answer 3

There is one way I think you could easily achieve a good approximation. You may first draw the silhouette on the texture as you already do:

Then, in a second step, add a frame around the texture with one pixel width (Note in the picture that the silhouette detached from the side of the window):

Finally, run the rest of your fragment shader code.

Manara