GLSL NVidia square artifacts

Question

I have encountered a problem when GLSL shader generates incorrect image on following GPU's:
GT 430
GT 770
GTX 570
GTX 760

But works normally on these:
Intel HD Graphics 2500
Intel HD 4000
Intel 4400
GTX 740M
Radeon HD 6310M
Radeon HD 8850

Shader code is as follows:

bool PointProjectionInsideTriangle(vec3 p1, vec3 p2, vec3 p3, vec3 point)
{
  vec3 n = cross((p2 - p1), (p3 - p1));

  vec3 n1 = cross((p2 - p1), n);
  vec3 n2 = cross((p3 - p2), n);
  vec3 n3 = cross((p1 - p3), n);

  float proj1 = dot((point - p2), n1);
  float proj2 = dot((point - p3), n2);
  float proj3 = dot((point - p1), n3);

  if(proj1 > 0.0)
    return false;
  if(proj2 > 0.0)
    return false;
  if(proj3 > 0.0)
    return false;
  return true;
}

struct Intersection
{
    vec3 point;
    vec3 norm;
    bool valid;
};

Intersection GetRayTriangleIntersection(vec3 rayPoint, vec3 rayDir, vec3 p1, vec3 p2, vec3 p3)
{
    vec3 norm = normalize(cross(p1 - p2, p1 - p3));

    Intersection res;
    res.norm = norm;
    res.point = vec3(rayPoint.xy, 0.0);
    res.valid = PointProjectionInsideTriangle(p1, p2, p3, res.point);
    return res;
}

struct ColoredIntersection
{
    Intersection geomInt;
    vec4 color;
};

#define raysCount 15
void main(void)
{
    vec2 radius = (gl_FragCoord.xy / vec2(800.0, 600.0)) - vec2(0.5, 0.5);

    ColoredIntersection ints[raysCount];

    vec3 randomPoints[raysCount];
    int i, j;


    for(int i = 0; i < raysCount; i++)
    {
        float theta = 0.5 * float(i);
        float phi = 3.1415 / 2.0;
        float r = 1.0;
        randomPoints[i] = vec3(r * sin(phi) * cos(theta),  r * sin(phi)*sin(theta), r * cos(phi));

        vec3 tangent = normalize(cross(vec3(0.0, 0.0, 1.0), randomPoints[i]));
        vec3 trianglePoint1 = randomPoints[i] * 2.0 + tangent * 0.2;
        vec3 trianglePoint2 = randomPoints[i] * 2.0 - tangent * 0.2;

        ints[i].geomInt = GetRayTriangleIntersection(vec3(radius, -10.0), vec3(0.0, 0.0, 1.0), vec3(0.0, 0.0, 0.0), trianglePoint1, trianglePoint2);
        if(ints[i].geomInt.valid)
        {
            float c = length(ints[i].geomInt.point);
            ints[i].color = vec4(c, c, c, 1.0);
        }
    }

    for(i = 0; i < raysCount; i++)
    {
        for(j = i + 1; j < raysCount; j++)
        {
            if(ints[i].geomInt.point.z < ints[i].geomInt.point.z - 10.0)
            {
                gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
                ColoredIntersection tmp = ints[j];
                ints[j] = ints[i];
                ints[i] = tmp;
            }
        }
    }

    vec4 resultColor = vec4(0.0, 0.0, 0.0, 0.0);
    for(i = 0; i < raysCount + 0; i++)
    {
        if(ints[i].geomInt.valid)
            resultColor += ints[i].color;
    }

    gl_FragColor = clamp(resultColor, 0.0, 1.0);
}

Upd: I have replaced vector normalizations with builtin functions and added gl_FragColor claming just in case.

The code is a simplified version of an actual shader, expected image is:

But what I get is:

Random rotations of the code remove artifacts completely. For example if I change the line

if(ints[i].geomInt.valid) //1

to

if(ints[i].geomInt.valid == true) //1

which apparently should not affect logic in any way or completely remove double cycle that does nothing (marked as 2) artifacts vanish. Please note that the double cycle does nothing at all since condition

if(ints[i].geomInt.point.z < ints[i].geomInt.point.z - 10.0)
{
  gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
  return;
  ColoredIntersection tmp = ints[j];
  ints[j] = ints[i];
  ints[i] = tmp;
}

Can never be satisfied(left and right sides have index i, not i, j) and there's no NaN's. This code does absolutely nothing yet somehow produces artifacts.

You can test the shader and demo on your own using this project(full MSVS 2010 project + sources + compiled binary and a shader, uses included SFML): https://dl.dropboxusercontent.com/u/25635148/ShaderTest.zip

I use sfml in this test project, but that's 100% irrelevant because the actual project I've enountered this problem does not use this lib.

What I want to know is why these artifacts appear and how to reliably avoid them.

Georgiannageorgianne

Interesting artifacts. On the setups the problem reproduces, is it completely repeatable? Say you run it 10 times in a row, do you always get the artifacts? Always exactly the same bad output, or is there some randomness to it? – Tellurian

It is 100% repeatable on most modern NVidia GPU's. If I run the shader on Intel built-in graphics there's no artifacts, if on gf640m, they appear. The square artifacts themselves change each frame without any noticable pattern. – Georgiannageorgianne

If you remove the return from your branch, does that change anything? – Alissaalistair

I have added that return as a debug measure to show that the condition is never true. And no, I have double-checked and that return does not affect result image in any way. – Georgiannageorgianne

Ah, this basically reminds me of unsynchronized read/write to the color buffer. You can produce an almost identical effect using image load/store without a memory barrier. However, since some of those blocks on the screen (particularly in the upper-left corner) are not squarely aligned to the window, I think this is an issue with computation rather than storage. Have you tried using pragmas to disable compiler optimizations? NV has quite a few proprietary ones, lookup #pragma optionNV. – Alissaalistair

You might also want to use built-in functions for things like normalization to avoid any hard to track FP issues. – Alissaalistair

But isnt any optimization supposed to optimize code without changing its logic? The only case when optimization can cause code to behave differently is when the code has any kind of undefined behaviour. So why should I disable any optimization if I actually want my code optimized(shader is actually quite costly)? And thanks for built-in normalization, I have forgotten about it. – Georgiannageorgianne

Not necessarily, optimization can do funny things with floating-point arithmetic. GPUs do not adhere completely to IEEE 754, with older models even less so. Rendering is far more finicky when it comes to this sort of stuff than a general purpose application running on the CPU. Compiling general software with fast math vs. accurate FP often will not show any difference at all, but compiling shaders with fast vs. accurate FP can have a world of difference. And the more you implement certain fundamental operations yourself instead of using built-ins, the more likely you are to run into this. – Alissaalistair

I have replaced all normalizations with builtin functions and added a safeguard - clamping for gl_FragColor. No effect on artifacts. Uploaded updated demo and changed source in the original post. – Georgiannageorgianne

Actually, using constants like 1e-5 can definitely lead to such a case. Depending on the precision of the floating-point representation used, these magic numbers may not even be representable. lowp, for instance, guarantees nothing smaller than 2^-8. – Alissaalistair

Answer 1

If anyone's still interested I asked this question on numerous specialized sites including opengl.org and devtalk.nvidia.com. I did not receive any concrete answer on what's wrong with my shader, just some suggestions how to work around my problem. Like use if(condition == true) instead of if(condition), use as simple algorithms as possible and such. In the end I've chosen one of the easiest rotations of my code that gets rid of the problem: I just replaced

struct Intersection  
{  
    vec3 point;  
    vec3 norm;  
    bool valid;  
};  

with

struct Intersection  
{  
    bool valid;  
    vec3 point;  
    vec3 norm;  
};  

There were numerous other code rotations that made the artifacts disappear, but I've chosen this one because I was able to test in on most other systems I had trouble with before.

Georgiannageorgianne

Honestly, I would have tried putting bool valid sandwiched in-between the two vec3s knowing what I do about GPUs' preferred data alignment. vec3 likes to begin on a 16-byte boundary, and is basically treated the same as a vec4. You can cram a 4-byte variable (yes, bool is 32-bit in GLSL) in-between two vec3s quite effectively without disrupting alignment or wasting space. – Alissaalistair

Answer 2

I don't think anything is wrong with your shader. The openGL pipeline renders to a framebuffer. If you make use of that framebuffer before the rendering has completed, you will often get what you have seen. Please bear in mind that glDrawArrays and similar are asynchonous (the function returns before the GPU has finished drawing the vertices.)

The most common use for those square artefacts are when you use the resultant framebuffer as texture which is then use for further rendering.

The OpenGL driver is supposed to keep track of dependencies and should know how to wait for dependencies to be fulfilled.

If you are sharing a framebuffer across threads, however, all bets are off, you then might need to make use of things like a fence sync (glFenceSync) to ensure that that one thread waits for rendering that is taking place on another thread.

As a workaround, you might find calling glFinish or even glReadPixels (with one pixel) sorts the issue out.

Please also bear in mind that this problem is timing related and simplifying a shader might very well make the issue go away.

Philippeville

Nope, syncing is not the reason. At least I failed to remove the artifacts by any syncing options(glFlush()/glFenceSync()/wglSwapInterval(),Sleep()). And I'm rendering directly to the window so render-to-texture sync issues are irrelevant as well. If you don't believe me you can easily test it yourself if you have windows and Nvidia GPU: all sources and libs are included in the attachment. – Georgiannageorgianne

Well then it could well be a bug in the NVidia OpenGL drivers. – Philippeville

Answer 3

I've seen this exact thing happen in GLSL when variables aren't initialized. For example, a vec3 will be (0,0,0) by default on some graphics cards, but on other graphics cards it will be a different value. Are you sure you're not using a variable without first assigning it a value? Specifically you aren't initializing ColoredIntersection.color if Insersection.valid is false, but I think you are using it later.

Ordway