is there a good robust algorithm to calculate normal vector of a convex polygon (in 3D, of course)? For triangles, it is easy: one takes two of the triangle's edges and calculates the cross product:
vec3 u = point[0] - point[1], v = point[0] - point[2];
vec3 n = normalize(cross(u, v));
But this approach does not really extend to polygons very well. Some edges of the polygon can be nearly or "exactly" collinear (this will happen often in meshes where T-Junction removal took place), therefore it is necessary to choose a pair of edges, giving a "strong" normal (both edges are "long enough" and they hold "almost perpendicular" angle).
This approach will still not work for all polygons, though. Imagine a polygon in the shape of a disc. If it is very finely subdivided, all the edges will be very short and all of the consecutive edges will be almost collinear, regardless of the radius of the disc. At the same time, the normal is very well defined.
One solution could be to find the largest inscribed triangle and to calculate the normal of that. However, finding it will have complexity of O(n^2)
, which seems prohibitive.
A better solution could be to use SVD or Eigenvalue decomposition to calculate the normal, given all the polygon points, not just three or four.
Is there a standard algorithm for this? Anyone has a good way of doing it?