Off-axis projection with glFrustum

Question

I am trying to do an off-axis projection of a scene with OpenGL and I gave a read to the document to Robert Kooima's off-axis projection and have a much better idea now of what actually has to be done but there are still some pieces which I am finding tricky here. I got to know of the off-axis projection code for OpenGL to be somewhat as follows:

Code 1:

glMatrixMode(GL_PROJECTION);  
    glLoadIdentity();            
    glFrustum(fNear*(-fFov * ratio + headX),  
              fNear*(fFov * ratio + headX),  
              fNear*(-fFov + headY),  
              fNear*(fFov + headY),  
              fNear, fFar);  
          
    glMatrixMode(GL_MODELVIEW);  
    glLoadIdentity();  
    gluLookAt(headX*headZ, headY*headZ, 0, headX*headZ, headY*headZ, -1, 0, 1, 0);
    glTranslatef(0.0,0.0,headZ);

Had this been a normal perspective projection with the user at the center of the screen, it is fairly easy to understand as I comprehend.

               Screen  
                   |
                   |  h = H/2
                   |  
x----- n -----------
                   |
                   |  h = H/2
                   |

With the user at x and distance from screen being n, the top, bottom coordinates for glFrustum would be calculated as: (assume theta is the Field of View (fov) which I suppose is assumed as 30 degrees)

h = n * tan (theta/2);
tanValue = DEG_TO_RAD * theta/2;
[EDIT Line additon here>>]: fFov = tan(tanValue);
h = n * tan (tanValue);

Hence, top and bottom (negating top's value) are both obtained for glFrustum arguments. Left one's are left/right for now.

Now, Aspect Ratio, r = ofGetWidth()/ofGetHeight();
Right = n * (fFov * r); , where r is the aspect ratio [Edit1>> Was written tanValue*r earlier here]

Question 1) Is the above (tanValue*r) getting the horizontal fov angle and then applying the same to get left/right value?

   double msX = (double)ofGetMouseX();
   double msY = (double)ofGetMouseY();
   double scrWidth = (double)ofGetWidth();
   double scrHeight = (double)ofGetHeight();

   headX = (msX / scrWidth) - 0.5;
   headY = ((scrHeight - msY) / scrHeight) - 0.5;
   headZ = -2.0;

Now, consider the projection off-axis and we have the headX and headY position computed (using mouse here instead of actual user's head):

Question 2) How is the headX and y being computed and what is the use subtracting -0.5 from the above? I observed that it brings the x-value to (-0.5 to 0.5) and y-value to (0.5 to -0.5) with msX and msY varying.

Question 3) In the above code (Code 1), how is headY being added to the calculated to the tan(fov/2) value?

-fFov + headY
fFov + headY

What does this value provide us with? -fFov was the calculated tan of theta/2 but how can headY be added to directly?

-fFov * ratio + headX
-fFov * ratio + headX

How does the abvoe give us a vlaue which wehn multiplied by n (near value) gives us left and right for the assymetric glFrustum call for off-axis projection?

Question 4) I understand that the glLookAt has to be done for View Point to shift the apex of the frustum to where the eye of the user is (in this case where the mouse is). Notice the line in the above code:

gluLookAt(headX*headZ, headY*headZ, 0, headX*headZ, headY*headZ, -1, 0, 1, 0);

How is headX*headZ giving me the xPosition of the eye, headY*headZ giving me the yPosition of the eye which I can use in gluLookAt() here?

EDIT: Full problem description added here: pastebin.com/BiSHXspb

Mychal

Answer 1

You have made this nice picture of ASCII art

               Screen  
                   B
                   |  h = H/2
                   |  
x----- n ----------A
                   |
                   |  h = H/2
                   B'

The field of view is defined as the angle fov = angle((x,B), (x,B')) formed between the two tips B, B' of the screen "line" and the point x. The trigonometric function Tangens (tan) is defines as

h/n = tan( angle((x,A), (x,B)) )

And since length(A, B) == length(A, B') == h == H/2 we know that

H/(2·n) == tan( fov ) == tan( angle((x,B), (x,B')) ) == tan( 2·angle((x,A), (x,B)) )

Since in trigonometry angles are given in radians, but most people are more comfortable with degrees you may have to convert from degress to radians.

So we're interested in only half of the screen span (= h) we've to half the angle. And if we want to accept degress also convert it to radians. That's what this expression is meant for.

tanValue = DEG_TO_RAD * theta/2;

Using that we then calculate h by

h = tan(tanValue) * n

If the FOV is for horizontal or vertical span of the screen depends on the way how the field span H is scaled with the aspect ratio.

How is the headX and y being computed and what is the use subtracting -0.5 from the above? I observed that it brings the x-value to (-0.5 to 0.5) and y-value to (0.5 to -0.5) with msX and msY varying.

The calculations you gave assume that screen space coordinates are in a range [0, screenWidth] × [0, screenHeight]. However since we're doing our frustum calculations in a normalized range [-1, 1]² we want to bring the device absolute mouse coordinates to normalized center relative coordinates. This allows then to specify the axis offset relative to the normalized near plane size. This is how it looks with 0 offset (the grid has 0.1 units distance in this picture):

And with a X offset of -0.5 applied it looks like this (orange outline), as you can see the left edge of the near plane has been shifted to -0.5.

Now simply imagine that the grid was your screen, and your mouse pointer would drag around the projection frustum near plane bounds like that.

What does this value provide us with? -fFov was the calculated tan of theta/2 but how can headY be added to directly?

Because fFov is not an angle but the span H/2 = h in your ASCII art picture. And headX and headY are relative shifts in the normalized near projection plane.

How is headXheadZ giving me the xPosition of the eye, headYheadZ giving me the yPosition of the eye which I can use in gluLookAt() here?

The code you're quoted seems to be an ad-hoc solution on that account to emphase the effect. In a real head tracking stereoscopic system you do slightly different. Technically headZ should be either used to calculated the near plane distance or be derived from it.

Anyway the main ideas is, that the head is located at some distance from the projection plane, and the center point is shifted in relative units of the projection. So you must scale relative headX, headY with the actual head distance to the projection plane to make the apex correction work.

Update due to comment/request

So far we've looked at only one dimension when converting field of view (fov) to screen span. For the image to be undistorted the aspect ratio of the [left, right] / [bottom, top] extents of the near clipping plane must match the aspect ratio of the viewport width/height.

If we choose to define the FoV angle to be the vertical FoV, then the horizontal size of the near clipping plane extents is the size of the vertical near clipping plane extents scaled with the with/height aspect ratio.

This is nothing special about off-axis projection, but can be found in every perspective projection helper function; compare the source code of gluPerspective for reference:

void GLAPIENTRY
gluPerspective(GLdouble fovy, GLdouble aspect, GLdouble zNear, GLdouble zFar)
{
   GLdouble xmin, xmax, ymin, ymax;

   ymax = zNear * tan(fovy * M_PI / 360.0); // M_PI / 360.0 == DEG_TO_RAD
   ymin = -ymax;

   xmin = ymin * aspect;
   xmax = ymax * aspect;

   glFrustum(xmin, xmax, ymin, ymax, zNear, zFar);
}

And if we consider the near clipping plane extents to be [-aspect, aspect]×[-1, 1] then of course the headX position is not in the normalized range [-1, 1] but must be given in the range [-aspect, aspect] as well.

If you look at the paper you linked, you'll find that for each screen the head position as reported by the tracker is transformed in absolute coordinates relative to the screen.

---

Two weeks ago I had the opportunity to test a display system called "Z space" where a polarized stereo display had been combined with a head tracker creating an off-axis frustum / look-at combination that matched your physical head position in front of the display. It also offers a "pen" to interact with the 3D scene in front of you. This is one of the most impressive things I've seen in the last few years and I'm currently begging my boss to buy us one :)

Unconscious

Thank you so much replying. I have been giving a thought to each and every small portion of 3D graphics and didn't just want to go ahead with the code itself. Had to read up a lot about matrices and other topics but there were some gaps remaining. This answer is going to become a reference for me. Thank you for so much effort:) – Mychal

The ASCII art figure was a sideways depiction of the screen and the user (x). The angle in degrees ( angle((x,A), (x,B)) ) as you pointed out becomes tanValue = DEG_TO_RAD * theta/2; in radians and subsequently fFov = tan(tanValue). So for Question 1 in the arguments of left/right in glFrustum, what is the meaning of fFov * ratio. Since I need to get left and right extents for the off-axis projection, I am not sure sure we are extracting x positions using fFov * ratio. Can you please expound a little on this as well? – Mychal

Consider the user's position as this -> bit.ly/10H0HLh. As per this digram screenHeight=768, mouseYPosition=668. So as per logic in my post above headY=(768-668)/768 - 0.5 = -0.36. That means when the user's head position (since mouseY is 668) is somewhere in the fourth quadrant as the displaced centre, the headY shift comes out to be -0.36. Subsequently, bottom value for glFrustum becomes (-fFov +headY) ~= -0.5 - 0.36 = -0.86, top = 0.5-0.36 = 0.14. (contd,) – Mychal

^(contd.) This point with top=0.14, bottom=-0.86 seems to be forming in the first quadrant rather than in the fourth. Why is that although top should be coming out to be 0.5+ and bottom to be coming out as -0.5- That is why I asked (Question 3) how could we add headY directly to fFov and still would appreciate a little bit more clarity on this, although I am getting the idea. – Mychal

@user1240679: I added a small update. Implementing such a kind of screen tracker with a Kinect has been on my agenda for a long time; I have not got around doing it yet; unfortunately it's a little bit more than just a weekend project to implement a calibrated head tracker with a Kinect; the OpenGL projection setup is rather easy though. – Unconscious

Thanks for the reply again. Sorry if I am stretching this a bit more but I wanted to understand the thing about normalised coordinates. I tried a separate test application in which I tried clipping a picture using glFrustum and the full picture was coming onto the screen when I used the following arguments: glFrustum(-2.58, 2.58, -1.5, 1.5, 3.0, 600.0); . These arguments for glFrustum are not restricted to [-1,1] but here up in the post as you mentioned that we are doing the frustum calculation in normalised range [-1,1]. – Mychal

What kind of arguments does glFrustum accept if not screen coordinates obviously. How do I establish the correspondence between screen coordinates and these glFrustum arguments, for e.g.: If I have a screen on the right part of the screen and want to defined wdith/2, height/2) on screen coordinates as the viewing volume, how would these be changed to glFrustum arguments? – Mychal