Why transform normals with the transpose of the inverse of the modelview matrix?

Question

I am working on some shaders, and I need to transform normals.

I read in few tutorials the way you transform normals is you multiply them with the transpose of the inverse of the modelview matrix. But I can't find explanation of why is that so, and what is the logic behind that?

Degree

Answer 1

Take a look at this tutorial:

https://paroj.github.io/gltut/Illumination/Tut09%20Normal%20Transformation.html

You can imagine that when the surface of a sphere stretches (so the sphere is scaled along one axis or something similar) the normals of that surface will all 'bend' towards each other. It turns out you need to invert the scale applied to the normals to achieve this. This is the same as transforming with the Inverse Transpose Matrix. The link above shows how to derive the inverse transpose matrix from this.

Also note that when the scale is uniform, you can simply pass the original matrix as normal matrix. Imagine the same sphere being scaled uniformly along all axes, the surface will not stretch or bend, nor will the normals.

Hyder

Answer 2

It flows from the definition of a normal.

Suppose you have the normal, N, and a vector, V, a tangent vector at the same position on the object as the normal. Then by definition N·V = 0.

Tangent vectors run in the same direction as the surface of an object. So if your surface is planar then the tangent is the difference between two identifiable points on the object. So if V = Q - R where Q and R are points on the surface then if you transform the object by B:

V' = BQ - BR
   = B(Q - R)
   = BV

The same logic applies for non-planar surfaces by considering limits.

In this case suppose you intend to transform the model by the matrix B. So B will be applied to the geometry. Then to figure out what to do to the normals you need to solve for the matrix, A so that:

(AN)·(BV) = 0

Turning that into a row versus column thing to eliminate the explicit dot product:

[tranpose(AN)](BV) = 0

Pull the transpose outside, eliminate the brackets:

transpose(N)*transpose(A)*B*V = 0

So that's "the transpose of the normal" [product with] "the transpose of the known transformation matrix" [product with] "the transformation we're solving for" [product with] "the vector on the surface of the model" = 0

But we started by stating that transpose(N)*V = 0, since that's the same as saying that N·V = 0. So to satisfy our constraints we need the middle part of the expression — transpose(A)*B — to go away.

Hence we can conclude that:

 transpose(A)*B = identity
 => transpose(A) = identity*inverse(B)
 => transpose(A) = inverse(B)
 => A = transpose(inverse(B))

Worry

@NicolBolas V is, as I said, a vector on the surface, not a location. So, yes, N.V is the dot product. I could equally have phrased it "a vector along the surface". As you point out, alternative interpretations are immediately ruled out through the formal mention of the dot product. – Worry

(though technically "a vector along the surface" would be less accurate since it presupposes the surface is flat; I'll edit to explicitly mention the word 'tangent') – Worry

@NicolBolas on the contrary, "You intend to transform the model by the matrix A" seems quite clear to me. Your question makes the implicit point that I'd verbally described A and B the wrong way around though. – Worry

One more question. If B is the model transform matrix, how do we know that the tangent V needs to be multiplied by it and not the normal N? – Transfix

@NicolBolas good point. I mean, you and I both know it's because one definition of the tangent is the limit between the difference between two points that are h units apart as h goes towards zero but it's not a complete answer unless I edit that in there. – Worry

Answer 3

My favorite proof is below where N is the normal and V is a tangent vector. Since they are perpendicular their dot product is zero. M is any 3x3 invertible transformation (M^-1 * M = I). N' and V' are the vectors transformed by M.

To get some intuition, consider the shear transformation below.

Note that this does not apply to tangent vectors.

Bellyful

There's a typo on the last one. It should have said $(M^-T N)^T MV = 0$ (transpose is missing around the produce of the first two matrices). – Unquestioned

I know this is an old thread...but how do you know that MV is the same as V'? I mean, the whole point here is that the transform matrix M doesn't work on N to make a valid N', so how can we assume that it works on the tangent V to make a valid V'? – Anderaanderea

V’ = MV by definition. i.e. M is the transform applied to the vertex positions. The tangent vector can be thought of as the difference of two points in the plane of the surface… those two points are transformed by M and thus the difference of the transformed points will lie in the transformed plane. – Bellyful

Okay. And even if the point is on a sphere or other curve, you can still define this vector V using an infinitesimal difference between 2 points, using a limit. I think I can buy that. (Thank you for revisiting an old thread and filling this in. It's appreciated.) – Anderaanderea

Answer 4

If the model matrix is made of translation, rotation and scale, you don't need to do inverse transpose to calculate normal matrix. Simply divide the normal by squared scale and multiply by model matrix and we are done. You can extend that to any matrix with perpendicular axes, just calculate squared scale for each axes of the matrix you are using instead.

I wrote the details in my blog: https://lxjk.github.io/2017/10/01/Stop-Using-Normal-Matrix.html

Peddle

For any orthonormal transformation M (e.g., rotation) the inverse-tranpose M^(-T) = M. Uniform scaling will only change the magnitude of the normal, not the direction. Translation as the last operation doesn't effect the upper 3x3 part so its moot. Everything else you should be using the inverse-transpose -- why not use the most general solution that works in all cases? – Bellyful

Surely you don't need to do anything special for uniform scale. For non-uniform scale this method is FASTER then sending over a normal matrix to shader or doing inverse transpose in shader. For model matrix this is a common case. If the model matrix does not meet the condition then unfortunately we have to fall back to inverse transpose. – Peddle

To compute one 3x3 inverse transpose to transform perhaps thousands of vertex normals is inconsequential. A non-inform scale following a rotation will not preserve normal directions. – Bellyful

That's 2 instructions (mul, div) versus 8 instructions (3 cross, 1 dot, 3 mul, 1 div), more than 4 times faster. (Both method need a matrix multiply at the end, so ignore that part) How important this is depends on your application. Correctness is proved in my blog post, read it if you are interested, it is exactly the same result as 3x3 inverse transpose. – Peddle

Transforming normals with a 3x3 matrix is one matrix-vector multiply (3 dot products in GLSL) -- cost of 3 multiplies (at most) on a SIMD GPU. Most of the cost comes when you renormalize the resulting vector after the multiply - GPU's are particularly fast at that (dot, rsqrt, dot) -- rsqrt is probably the most expensive, but there are tricks for that and will be done in silicon (h14s.p5r.org/2012/09/0x5f3759df.html) – Bellyful

I liked your blog, but are you computing the inverse transform for every vertex? I usually don't do that in the vertex shader -- I pre-compute the normal matrix (only needed if model or view matrix change) and pass it to the shader. – Bellyful

I used to do that as well, calculate normal matrix per geometry on CPU and send it over to shader. Then I find out this way, which is cheap enough that I can afford it to be calculated in vertex shader. It will save me from sending an extra matrix to shader per geometry, which I think is a win :) But yeah as you said it won't work for general matrix, so it is trade-off decision. – Peddle

Answer 5

Don't understand why you just don't zero out the 4th element of the direction vector before multiplying with the model matrix. No inverse or transpose needed. Think of the direction vector as the difference between two points. Move the two points with the rest of the model - they are still in the same relative position to the model. Take the difference between the two points to get the new direction, and the 4th element, cancels out to zero. Lot cheaper.

Gwenora

Modelview matrices may also scale, including in different amounts in different directions, not necessarily oriented with axes. i.e. the top left 3x3 is not necessarily orthonormal. If you limit yourself to special orthogonal matrices then your solution is sufficient. – Worry