Shadow Mapping - artifacts on thin wall orthogonal to light

Question

I'm having an issue with back faces (to the light) and shadow mapping that I can't seem to get past. I'm still at the relatively early stages of optimizing my engine, however I can't seem to get there as even with everything hand-tuned for this one piece of geometry it still looks like garbage.

What it is is a skinny wall that is "curved" via about 5 different chunks of wall. When I create my depth map I'm culling front faces (to the light). This definitely helps, but the front faces on the other side of the wall are what seem to be causing the z-fighting/projective shadowing.

Some notes on the screenshot:

• Front faces are culled when the depth texture (from the light) is being drawn
• I have the near and far planes tuned just for this chunk of geometry (set at 20 and 25 respectively)
• One directional light source, coming down on a slight angle toward the right side of the scene, enough to indicate that wall should be shadowed, but mostly straight down
• Using a ludicrously large 4096x4096 shadow map texture
• All lighting is disabled, but know that I am doing soft lighting (and hence vertex normals for the vertices) even on this wall

As mentioned here it concludes you should not shadow polygons that are back faced from the light. I'm struggling with this particular issue because I don't want to pass the face normals all the way through to the fragment shader to rule out the true back faces to the light there - however if anyone feels this is the best/only solution for this geometry thats what I'll have to do. Considering how the pipeline doesn't make it easy/obvious to pass the face normals through it makes me feel like this isn't the path of least resistance. And note that the normals I am passing are the vertex normals, to allow for softer lighting effects around the edges (will likely include both non-shadowed and shadowed surfaces).

Note that I am having some nasty Perspective Aliasing, but I'm hoping my next steps are to work on cascaded shadow maps, but without fixing this I feel like I'm just delaying the inevitable as I've hand-tightened the view as best I can (or so I think).

Anyways I feel like I'm missing something, so if you have any thoughts or help at all would be most appreciated!

EDIT

To be clear, the wall technically should NOT be in shadow, based on where the light is coming from.

Below is an image with shadowing turned off. This is just using the vertex normals to calculate diffuse lighting - its not pretty (too much geometry is visible) but it does show that some of the edges are somewhat visible.

So yes, the wall SHOULD be in shadow, but I'm hoping I can get the smoothing working better so the edges can have some diffuse lighting. If I need to have it completely in shadow, then if its the shadow map that puts it in shadow, or my code specifically putting it in shadow because the face normal is away, I'm fine with that - but passing the face normal through to my vertex/fragment shader does not seem like the path of least resistance.

Perhaps these will help illustrate my problem better, or perhaps bring to light some fundamental understanding I am missing.

EDIT #2

I've included the depth texture below. You can see the wall in question in the bottom left, and from the screenshot you can see how i've trimmed the depth values to ~0.4->1. This means the depth values of that wall start in the 0.4 range. So its not PERFECTLY clipped for it, but its close. Does that seem reasonable? I'm pretty sure its a full 24 or 32 bit depth buffer, a la DEPTH_COMPONENT extension on iOS. For @starmole, does this help to determine if its a scaling error in my projection? Do you think the size/area covered of my map is too large, hence if it focuses closer it might help?

Quadrille

Just to make sure I understand this properly, is this wall supposed to be shadowed or lit? Some annotations in the image to indicate the angle of the directional light might also be helpful. – Moreta

Added some edits, but to summarize - technically, based on the geometry, it should be in shadow... but my guess is using shadow maps to shadow faces on that steep an angle with the light is not advisable (because of the artifacts, ie erroneous self shadowing). – Quadrille

Perhaps I'm still missing something, then—if the portion of the wall facing the viewer is facing away from the light, shouldn't it already be dark? Hopefully there's no sharing of normals between the vertices that make up the top face of the wall and the viewer-facing part. Also, does the light's arrow indicate where the light is, or the direction in which the light is shining? – Moreta

You are correct - however in the first screenshot, there is no lighting of any kind. Perhaps my entire issue is academic, or actually a red herring. As you can see in the last, even those "hidden from light" walls do have some lighting on them. This is a result of my standard "vertex normal DOT light direction" to determine diffuse intensity. If I was using the face normal here, there is no doubt there would be no light on the wall... but as I said, passing that face normal through doesn't seem like the standard solution. – Quadrille

Regarding the light position - it is at (0, 25, 0), which puts it at that little pink circle in the bottom left corner of the overhead image. I think this only matters for the view matrix that is used for generating the depth map, but perhaps I am wrong about that. – Quadrille

Actually, I think you want to make you have correct normals on the pieces of the wall, even if that means duplicating the vertices at corners to ensure that each face gets the correct normal. Just calculating shadowing will give you a binary (in/out) result, rather than the shading I presume you're looking for. – Moreta

I think you have outlined the fundamental problem - geometry. I shouldn't have to pass the face normals through. I should structure the geometry so it indicates how it wants to be shaded, and then let the normal light calculations take hold. The shadow mapping artifacts are then irrelevant on these faces, as they SHOULD be in shadow controlled by the lighting. No fancy code needed, and only the typical shadow mapping workarounds (ie cull FRONT on depth map, and refinement of frustrum/shadow map size). THANK YOU for clearing this up for me! – Quadrille

And for the record I did as you said, fixed the vertex normals by duplicating those vertices, and the lighting has those faces in shadow. – Quadrille

Glad that worked out for you. Should I write an actual answer for the record? – Moreta

Yes, I think you should if thats ok! – Quadrille

Hm... not putting an answer either, but I think the comment thread is wrong. While it is true that the artifacts look like to close corner cases, and sometimes you need to work around those, with your parameters it just seems like a simple scaling error in your shadow projection. One good way to debug this is to re-project and just render eye depth derived from the shadow map only. – Marianelamariani

I threw in the depth texture from the light - hopefully it will help or bring to light any issues you think I have in scaling. – Quadrille

Hmm, that's a good point. If the directional light is shining at an angle rather than straight down, why does the depth texture only appear to be showing the top of the wall, rather than the sides? Can you show the matrices and shaders that you're using to render the depth texture and do the shadow comparison? – Moreta

I am really confused by your illustrations and text. Can you come up with a simpler test with only cubes and planes maybe? :) On fixed axis? Or does your code work then? I also agree with Pivot, that shadow map looks nothing like what I would expect to see. – Marianelamariani

Thanks for the comments - I'll get back to you guys in a few days with more info, i've torn all the code apart now for VSM and CSM so its a lot different. To answer a couple questions quickly, when i said slight angle for light, i really meant it. Its something like (0.05, -0.99, 0) or in that range, so i think the depth texture is ok. As for the simpler example, ya, everything in my original scene worked except this thin wall. I really think its a known "issue" that simply needs to be compensated for with proper lighting. – Quadrille

Hey guys - I'm working to reproduce this now, and I'm finding the changes I've made recently shift the problem a little. It seems I the original issue is mostly solved by glPolygonOffset(1.0, 4096.0). However, i've realized I culling front faces introduces other issues, i.e. on the depth draw I can't get nice blurring without the small variance in depth values for items on the ground (and hence wouldn't have any faces pushed to depth). When I re-introduce front faces to depth draw, I'm back where I started, if not worse. – Quadrille

So I think I really need to stick to the lighting solution… but I am curious about star mole's "scaling error in shadow projection". Can you give me an example? Like bad near/far planes? Also, I can definitely post all the shaders and matrices that are passed... but its a lot, just want to double check you guys are still interested, I feel like I've dragged this on for too long already :) – Quadrille

The problem with shadow mapping implementation is that in the end you will have case where it breaks and you need to deal with epsilons and tricks to make it better. So it is very hard to tell if those are the issue or if there is a more basic mistake. In my experience it really pays to look for the basic mistake first. So be systematic about that first: Render one sample, nearest filtering, output shadowz - lightz. You should see two errors: Projection as a mosaic pattern, precision as noise around 0. If your error is consistently positive or negative or biasd, you have a basic mistake. – Marianelamariani

Then make some test cases (like science!) where you know what you expect to see if everything would be right. Move a plane away from the light slowly, float error should go from positive to negative. Make sure that the error blows up exactly when you pass far and near distances. At this point it might be fun to change the way you encode depth values. How do things change with less precision? With storing z or 1/z? With computing 1/z with rcp or div? – Marianelamariani

Sorry for rambling :) But treat it as a science experiment and not as a hack. Especially once you go to CSM etc. It's a really interesting algorithm in that sense: In the end you will need to add epsilons to make it work. So it is tempting to look at them early. But it will really help you to spend time on understanding exactly why you need them. Make theory. Implement. Don't stop until you exactly match it or know how to change you theory. Repeat. After that epsilons and hacks are easy. Good luck :) – Marianelamariani

@Marianelamariani thanks so much for the thoughts, they really helped. Broken down to first principles, this wall is the only piece of geometry that breaks my engine. With the light facing STRAIGHT down, nearest filtering, 24bit depth values (in GL_DEPTH_COMPONENT texture), and no epsilons anywhere, the artifacts are present on this wall. I move the light up and down, and the near/far are clearly the extents, and the whole way through the artifacts are present. Other surfaces that I expect are orthogonal do not exhibit this behaviour. – Quadrille

From a theoretical point of view, what should I expect to see on faces that are orthogonal to the light direction if culling is disabled? I think based on the theory I should expect "erroneous self shadowing", which I think this is exactly what this... except in this case my guess is there is some slight skew in the geometry just enough to cause the depth values to not be smooth along the edges of the wall, and this leads to varying shadow tests, and leads to the artifacts. – Quadrille

Answer 1

The problem seems to be that you are

1. Culling the front faces
2. Looking at the back face
3. Not removing the light from the back face because it's actually not lit by the normal - or there is some inaccuracy in the computation
4. Probably not adding some epsilon

(1) and (2) mean that there will be Z-fighting between the shadow map and the back faces.

Also, the shadow map resolution is not going to help you - just look at the wall in the shadow map, it's one pixel thick.

Recommendations:

1. Epsilons. Make sure that Z > lightZ + epsilon
2. Epsilons. Make sure that the wall is facing the light (dot of normal > epsilon) to make sure the wall is shadowed if it's very nearly orthogonal

Brut

I am marking this as a solution, because technically it is - the main thing that solves this issue is the fact that these are technically backfaces to the light, and should be in full shadow (item #3). Epsilon if applied to the final values does not help this as it simply pushes the shadows down the -z direction. So the top of the wall looks ok, but the problem still starts somewhere. Regardless, thanks for the answer, hope it is able to help others. – Quadrille