Tangent, binormal and other shader-related things

Question

Many of the mapping techniques including normal bump mapping, parallax mapping and others require the special per-vertex tangent-space basis (tangent, normal, binormal / bitangent).

This obviously means that my models should not only export vertex positions, texture coordinates and approximated per-vertex normals, but also one of the tangent space basis vectors (usually tangent), because the other one can be found directly in the shader using cross(tangent, normal).

Note that position, normal, uv and tangents actually depend on each other the following way (you have to know everything else about the vertex to prepare the tangent basis).

position -> normal ->  tangents
            uv     ->

Now - how is this sort of things handled in modern 3D games / rendering engines?

Do they actually supply the normals, tangents and the uv coordinates per each vertex or can they somehow be calculated in runtime? Should they be the part of the model data or should they be a runtime-only property?

I also know that when using Direct3D10+ using the geometry shaders one can actually prepare the normals and the the tangents right in runtime (obviously, because we have access to the vertices in each triangle) - is it worth it or should these things be always precomputed?

Thaw

Answer 1

My view is as follows:

UV coordinates: clearly precomputed. UV unwrapping of a complex model is not a trivial, algorithmic task and usually involves manual placing of seams for best result. Also, if your model comes with a texture skin, then the UVs need to match the skin, so in this case they may not be computed during runtime.

Normals: theoretically you could precompute them in a geometry shader or even on CPU during loading. But there's one thing: the real models in games are often NOT the original models, but versions with lower polygon count. In the final model used in-game, you have less vertices, but the per-vertex normals are calcuated more accurately from the original high-poly model (which is also used for ambient occlusion calculation, etc). Therefore, having the normals precalculated is also feasible.

Also, in many games a texture with per-texel normal map (usually in tangent space, AFAIK) is provided together with the color texture for each model, for the purposes of parallel mapping later in the pixel shader.

Therefore it looks like my suggestion is clear - go for precomputed data, have better details and save on either shader processing or loading time - unless you need to worry about the model size (i.e. GPU memory limit) - only then having normals calculated in geometry shader would save you some space.

Adame

Answer 2

Generally you'll want to precompute as much as possible. Most engines will have a custom written tool that can take in raw data from 3D software, and massage it into a format for run time (adding in tangents and bitangents, etc). However...

In certain situations the per-vertex memory overhead can be a particular performance concern. This is because there's an overhead associated with moving data about the system -- it's "hidden" because it's not an explicit as an instruction added into a shader, so it's a little more advanced.

In these cases, the bitangent may not be precomputed, and instead be constructed at runtime (in order to reduce the size of the vertex in memory). Bitangents are appropriate for this because they can be built from a normal, tangent and a directionality flag -- which are all properties of a single vertex.

Normals, some UV mappings and other things require knowledge of the "topology" of the mesh -- and this topology isn't usually present in runtime data. They also require extra markings from the artists; like sharp edges and textures seams, etc. For this reason, normals, UVs and the first tangent will almost always be precalculated for all but very specialised types of geometry.

Remember that that the representation of a model in Max, Maya or Blender is very different than the run time version.

Editor software

• soup of triangles and quads, with neighbour information
• edges marked as sharp/soft
• a single vertex can have multiple normals and multiple UV coords

Run time (for DirectX, OpenGL, etc)

• cache friendly collection of triangle strips and triangle fans
• edges are implicit, there is no extra information tied to them
• each vertex has only one position, one normal and one UV coord (per texture)
• sometimes exists at multiple levels of detail

In general, you lose a lot of information that cannot be reconstructed. Your artists will be happiest if you do as much as you can with the original Max/Maya version of the geometry.

Sunshinesunspot

Answer 3

You can also do this with ID3DXMesh in D3D9. You should always compute them in advance- because it's wasteful to compute them again at runtime. However, for each mesh, you'd only ever compute them once, so unless your program procedurally generates meshes, it's probably a non-issue about where and when it's done.

The only important thing here is to compute them on the CPU, not the GPU, because you're doing that work again and again on the GPU every frame, which is a total waste compared to just once on the CPU. As far as I know, all 3D rendering systems compute the normals and such on the CPU once, save it to the file, and then load it and send it off to the GPU for work.

Mackenzie

Answer 4

I prefer to pre-compute them, as you need special handling with seams, mirrored textures, wraparounds and such.

It's not entirely trivial to compute it during runtime as you might need more than the current triangle data to get a correct tangent space.

Feasible

Answer 5

UVs and normals are usually under the control of the artist. Tangents and binormals can be calculated using the gradient of the UVs. It tends to be a bit expensive to calculate the tangent and binormals all the time, so you usually precalculate them based on the underlying geometry once. Preferably before you ship to reduce loading times.

Inexistent