How do I calculate pixel shader depth to render a circle drawn on a point sprite as a sphere that will intersect with other objects?

Question

I am writing a shader to render spheres on point sprites, by drawing shaded circles, and need to write a depth component as well as colour in order that spheres near each other will intersect correctly.

I am using code similar to that written by Johna Holwerda:

void PS_ShowDepth(VS_OUTPUT input, out float4 color: COLOR0,out float depth : DEPTH)
{
   float dist = length (input.uv - float2 (0.5f, 0.5f)); //get the distance form the center of the point-sprite
   float alpha = saturate(sign (0.5f - dist));
   sphereDepth = cos (dist * 3.14159) * sphereThickness * particleSize; //calculate how thick the sphere should be; sphereThickness is a variable.

   depth = saturate (sphereDepth + input.color.w); //input.color.w represents the depth value of the pixel on the point-sprite
   color = float4 (depth.xxx ,alpha ); //or anything else you might need in future passes
}

The video at that link gives a good idea of the effect I'm after: those spheres drawn on point sprites intersect correctly. I've added images below to illustrate too.

I can calculate the depth of the point sprite itself fine. However, I am not sure show to calculate the thickness of the sphere at a pixel in order to add it to the sprite's depth, to give a final depth value. (The above code uses a variable rather than calculating it.)

I've been working on this on and off for several weeks but haven't figured it out - I'm sure it's simple, but it's something my brain hasn't twigged.

Direct3D 9's point sprite sizes are calculated in pixels, and my sprites have several sizes - both by falloff due to distance (I implemented the same algorithm the old fixed-function pipeline used for point size computations in my vertex shader) and also due to what the sprite represents.

How do I go from the data I have in a pixel shader (sprite location, sprite depth, original world-space radius, radius in pixels onscreen, normalised distance of the pixel in question from the centre of the sprite) to a depth value? A partial solution simply of sprite size to sphere thickness in depth coordinates would be fine - that can be scaled by the normalised distance from the centre to get the thickness of the sphere at a pixel.

I am using Direct3D 9 and HLSL with shader model 3 as the upper SM limit.

In pictures

To demonstrate the technique, and the point at which I'm having trouble:

Start with two point sprites, and in the pixel shader draw a circle on each, using clip to remove fragments outside the circle's boundary:

One will render above the other, since after all they are flat surfaces.

Now, make the shader more advanced, and draw the circle as though it was a sphere, with lighting. Note that even though the flat sprites look 3D, they still draw with one fully in front of the other since it's an illusion: they are still flat.

(The above is easy; it's the final step I am having trouble with and am asking how to achieve.)

Now, instead of the pixel shader writing only colour values, it should write the depth as well:

void SpherePS (...any parameters...
    out float4 oBackBuffer : COLOR0,
    out float oDepth : DEPTH0 <- now also writing depth
   )
{

Note that now the spheres intersect when the distance between them is smaller than their radiuses:

How do I calculate the correct depth value in order to achieve this final step?

Edit / Notes

Several people have commented that a real sphere will distort due to perspective, which may be especially visible at the edges of the screen, and so I should use a different technique. First, thanks for pointing that out, it's not necessarily obvious and is good for future readers! Second, my aim is not to render a perspective-correct sphere, but to render millions of data points fast, and visually I think a sphere-like object looks nicer than a flat sprite, and shows the spatial position better too. Slight distortion or lack of distortion does not matter. If you watch the demo video, you can see how it is a useful visual tool. I don't want to render actual sphere meshes because of the large number of triangles compared to a simple hardware-generated point sprite. I really do want to use the technique of point sprites, and I simply want to extend the extant demo technique in order to calculate the correct depth value, which in the demo was passed in as a variable with no source for how it was derived.

Tropicalize

neat demo! it seems to me that the cos (dist * 3.14159) term is already varying the depth depending on the texture coordinate. that term evaluates to a range from -1 to 1. so, sphereThickness * particleSize must be the radius of your sphere (constant for drawing an individual sphere). – Jolynnjon

i'm pretty sure cos (dist * 3.14159) is the wrong formula, but maybe it works fine in practice. i think sqrt(1-4*dist*dist) is correct. (but, this change only fixes a more subtle problem) – Jolynnjon

It is a neat demo! And you're right, that line is varying the output depth based on the tex coordinate, i.e. the location of the pixel in the circle/sphere. But the demo left out or didn't implement the most important bit, which is calculating the size/radius in depth coordinates of the sphere itself - you can see it's just using preset variables sphereThickness and particleSize. – Tropicalize

As in, how do you go from the sprite size (in pixels, or in world coordinates) to what depth to write out for the centre, say, of the sphere - i.e. the depth of the sphere itself, or the thickness or radius of the sphere in depth coordinates that can be written to the DEPTH0 pixel shader output? – Tropicalize

Would an orthographic projection be OK? This makes the problem much simpler. If you want perspective.. will your scene need to depth buffer for anything besides the spheres? Perhaps the perspective could be faked by converting each sphere's position/radius from the perspective scene to an orthographic scene (e.g. the conversion would give a closer sphere a larger radius). The result wouldn't be exactly correct, but might look OK (?) – Jolynnjon

@TomSirgedas: No, I would prefer perspective not orthographic - the data is spatial (georeferenced) and needs to be positioned in space relative to other objects. That answers your other question too - there are other objects in the scene. – Tropicalize

Answer 1

I came up with a solution yesterday, which which works well and and produces the desired result of a sphere drawn on the sprite, with a correct depth value which intersects with other objects and spheres in the scene. It may be less efficient than it needs to be (it calculates and projects two vertices per sprite, for example) and is probably not fully correct mathematically (it takes shortcuts), but it produces visually good results.

The technique

In order to write out the depth of the 'sphere', you need to calculate the radius of the sphere in depth coordinates - i.e., how thick half the sphere is. This amount can then be scaled as you write out each pixel on the sphere by how far from the centre of the sphere you are.

To calculate the radius in depth coordinates:

• Vertex shader: in unprojected scene coordinates cast a ray from the eye through the sphere centre (that is, the vertex that represents the point sprite) and add the radius of the sphere. This gives you a point lying on the surface of the sphere. Project both the sprite vertex and your new sphere surface vertex, and calculate depth (z/w) for each. The different is the depth value you need.

• Pixel Shader: to draw a circle you already calculate a normalised distance from the centre of the sprite, using clip to not draw pixels outside the circle. Since it's normalised (0-1), multiply this by the sphere depth (which is the depth value of the radius, i.e. the pixel at the centre of the sphere) and add to the depth of the flat sprite itself. This gives a depth thickest at the sphere centre to 0 and the edge, following the surface of the sphere. (Depending on how accurate you need it, use a cosine to get a curved thickness. I found linear gave perfectly fine-looking results.)

Code

This is not full code since my effects are for my company, but the code here is rewritten from my actual effect file omitting unnecessary / proprietary stuff, and should be complete enough to demonstrate the technique.

Vertex shader

void SphereVS(float4 vPos // Input vertex,
   float fPointRadius, // Radius of circle / sphere in world coords
   out float fDXScale, // Result of DirectX algorithm to scale the sprite size
   out float fDepth, // Flat sprite depth
   out float4 oPos : POSITION0, // Projected sprite position
   out float fDiameter : PSIZE, // Sprite size in pixels (DX point sprites are sized in px)
   out float fSphereRadiusDepth : TEXCOORDn // Radius of the sphere in depth coords
{
    ...
   // Normal projection
   oPos = mul(vPos, g_mWorldViewProj);

   // DX depth (of the flat billboarded point sprite)
   fDepth = oPos.z / oPos.w;

   // Also scale the sprite size - DX specifies a point sprite's size in pixels.
   // One (old) algorithm is in http://msdn.microsoft.com/en-us/library/windows/desktop/bb147281(v=vs.85).aspx
   fDXScale = ...;
   fDiameter = fDXScale * fPointRadius;

   // Finally, the key: what's the depth coord to use for the thickness of the sphere?
   fSphereRadiusDepth = CalculateSphereDepth(vPos, fPointRadius, fDepth, fDXScale);

   ...
}

All standard stuff, but I include it to show how it's used.

The key method and the answer to the question is:

float CalculateSphereDepth(float4 vPos, float fPointRadius, float fSphereCenterDepth, float fDXScale) {
   // Calculate sphere depth.  Do this by calculating a point on the
   // far side of the sphere, ie cast a ray from the eye, through the
   // point sprite vertex (the sphere center) and extend it by the radius
   // of the sphere
   // The difference in depths between the sphere center and the sphere
   // edge is then used to write out sphere 'depth' on the sprite.
   float4 vRayDir = vPos - g_vecEyePos;
   float fLength = length(vRayDir);
   vRayDir = normalize(vRayDir);
   fLength = fLength + vPointRadius; // Distance from eye through sphere center to edge of sphere

   float4 oSphereEdgePos = g_vecEyePos + (fLength * vRayDir); // Point on the edge of the sphere
   oSphereEdgePos.w = 1.0;
   oSphereEdgePos = mul(oSphereEdgePos, g_mWorldViewProj); // Project it

   // DX depth calculation of the projected sphere-edge point
   const float fSphereEdgeDepth = oSphereEdgePos.z / oSphereEdgePos.w;
   float fSphereRadiusDepth = fSphereCenterDepth - fSphereEdgeDepth; // Difference between center and edge of sphere
   fSphereRadiusDepth *= fDXScale; // Account for sphere scaling

   return fSphereRadiusDepth;
}

Pixel shader

void SpherePS(
   ...
    float fSpriteDepth : TEXCOORD0,
    float fSphereRadiusDepth : TEXCOORD1,
    out float4 oFragment : COLOR0,
    out float fSphereDepth : DEPTH0
   )
{
   float fCircleDist = ...; // See example code in the question
   // 0-1 value from the center of the sprite, use clip to form the sprite into a circle
   clip(fCircleDist);    

   fSphereDepth = fSpriteDepth + (fCircleDist * fSphereRadiusDepth);

   // And calculate a pixel color
   oFragment = ...; // Add lighting etc here
}

This code omits lighting etc. To calculate how far the pixel is from the centre of the sprite (to get fCircleDist) see the example code in the question (calculates 'float dist = ...') which already drew a circle.

The end result is...

Result

Voila, point sprites drawing spheres.

Notes

• The scaling algorithm for the sprites may require the depth to be scaled, too. I am not sure that line is correct.
• It is not fully mathematically correct (takes shortcuts) but as you can see the result is visually correct
• When using millions of sprites, I still get a good rendering speed (<10ms per frame for 3 million sprites, on a VMWare Fusion emulated Direct3D device)

Tropicalize

Answer 2

The first big mistake is that a real 3d sphere will not project to a circle under perspective 3d projection.

This is very non intuitive, but look at some pictures, especially with a large field of view and off center spheres.

Second, I would recommend against using point sprites in the beginning, it might make things harder than necessary, especially considering the first point. Just draw a generous bounding quad around your sphere and go from there.

In your shader you should have the screen space position as an input. From that, the view transform, and your projection matrix you can get to a line in eye space. You need to intersect this line with the sphere in eye space (raytracing), get the eye space intersection point, and transform that back to screen space. Then output 1/w as depth. I am not doing the math for you here because I am a bit drunk and lazy and I don't think that's what you really want to do anyway. It's a great exercise in linear algebra though, so maybe you should try it. :)

The effect you are probably trying to do is called Depth Sprites and is usually used only with an orthographic projection and with the depth of a sprite stored in a texture. Just store the depth along with your color for example in the alpha channel and just output eye.z+(storeddepth-.5)*depthofsprite.

Sancha

Thanks starmole. I know real spheres will distort towards the edge of the screen, but this is a good technique for putting lots and lots (millions) of spheres onscreen at the cost of one vertex each. I'm afraid I'm not sure what you mean by drawing 'a generous bounding quad around the sphere and go from there'... isn't that what a sprite does, draws a quad? – Tropicalize

Re your algorithm: if it's 'not what you really want to do'... what is it for? I'm not sure, reading it, how it derives the depth value of the sphere as opposed to the sprite, either. Would you mind going into more detail, please? – Tropicalize

Ok, i'll put a drawing and write out the math in the evening. It would help if you could put the projection matrix you are using here. – Sancha

Here's code for a C++ proof-of-concept: pastebin.com/YwGhvzUd. And screenshot: screencast.com/t/iw0u04hI. The top-right quadrant of the window is drawn by a pixel shader. The rest is a sphere mesh. Note the rounder outline of the pixel shader version. – Jolynnjon

Answer 3

Sphere will not project into a circle in general case. Here is the solution.

This technique is called spherical billboards. An in-depth description can be found in this paper: Spherical Billboards and their Application to Rendering Explosions

You draw point sprites as quads and then sample a depth texture in order to find the distance between per-pixel Z-value and your current Z-coordinate. The distance between the sampled Z-value and current Z affects the opacity of the pixel to make it look like a sphere while intersecting underlying geometry. Authors of the paper suggest the following code to compute opacity:

float Opacity(float3 P, float3 Q, float r, float2 scr)
{
   float alpha = 0;
   float d = length(P.xy - Q.xy);
   if(d < r) {
      float w = sqrt(r*r - d*d);
      float F = P.z - w;
      float B = P.z + w;
      float Zs = tex2D(Depth, scr);
      float ds = min(Zs, B) - max(f, F);
      alpha = 1 - exp(-tau * (1-d/r) * ds);
   }
   return alpha;
}

This will prevent sharp intersections of your billboards with the scene geometry.

In case point-sprites pipeline is difficult to control (i can say only about OpenGL and not DirectX) it is better to use GPU-accelerated billboarding: you supply 4 equal 3D vertices that match the center of the particle. Then you move them into the appropriate billboard corners in a vertex shader, i.e:

if ( idx == 0 ) ParticlePos += (-X - Y);
if ( idx == 1 ) ParticlePos += (+X - Y);
if ( idx == 2 ) ParticlePos += (+X + Y);
if ( idx == 3 ) ParticlePos += (-X + Y);

This is more oriented to the modern GPU pipeline and of coarse will work with any nondegenerate perspective projection.

Doorsill

OP wants opaque spheres, not translucent – Jolynnjon

"sample a depth texture in order to find the distance between per-pixel Z-value and your current Z-coordinate... affects the opacity" - that means reading the depth of a texture representing the existing scene, and blending based on that? I'm after opaque spheres, and also am trying to find the correct Z for the 'sphere' in the first place - no need to compare to the depth of the existing scene objects, since DX won't write the fragment if the depth test fails. The 'per-pixel Z-value' you refer to is what I need to calculate I think. I came up with a solution - have a look at my answer. – Tropicalize