Getting accurate collision shapes for heightmap generated by a shader

Question

I have a Visual Shader displacing a plane mesh using a NoiseTexture2D Simplex noise. Visually I got the basics looking alright. However I'm not entirely sure how to do collisions, with support for other things later such as a potential occlusion mesh or pathfinding and of course grass particles and so on.

The issue is I'm using a noise texture resource, which I may modify in other complex ways throughout the shader. The noise function in GDScript may follow the same base patterns, but it wouldn't be the exact same texture and its coordinates. Is it possible to use the output of a vertex shader as height data for a HeightMapShape3D and other things that have to be set via script?

Hodometer

Answer 1

So you may have better luck with pre-computing it using MeshDataTool. You can still use noise or heightmaps, but it would be with GDScript, not shader code. This would keep it on the CPU.

This is fine for procedural generation at load time, but cannot be modified or animated in real-time. But for most things, this is a better method and will perform better.

Otherwise you would have to read the data back from the GPU to the CPU. In Godot 3.x, I don't think this is possible without doing tricks like baking to a texture.

In Godot 4.0 you can access uniform buffers in the shader, so it may be possible to read this back to the CPU. I haven't tried it yet.

Goosander

Answer 2

Initially I was thinking of infinite heightmap terrain that goes on as you explore, possibly limited but at least very large island. I kinda gave up that idea upon realizing that using a noise texture is finite, since unlike Blender Godot bakes it into an image thus it gets repetitive after a while: Only way it's infinite and unique is by using GDscript which lets you read noise values at any location without boundaries. There's probably a custom shader to generate truly infinite noise directly from it but I didn't look that much.

Biggest reason I wanted to try shader modified heightmaps is the ability to better correlate the mesh with the textures. Like say having a dirt road which also digs into the surface by decreasing the height. I could just use a blend of the two methods, where the vertex shader only does small offsets which don't need to modify the collision, not sure how that would work out.

Still it would be good to know: If you have a noise texture image resource, does GDScript allow you to read its pixel values at a certain global position? I believe I bumped into a guide on that but it said it was for exr images only, I imagine there's a way for the procedural noise too. The shader does many modifications however so they still wouldn't line up properly.

Hodometer

Answer 3

Yes, you can use get_pixel() on an image in GDScript, that's what I was implying. You should be able to get infinite noise, if you want, though you may need to recalculate the noise when you spawn new chunks.

Goosander

Answer 4

I actually do collision tests against a surface triangle mesh, but that approach, though relatively exact, is too much computation for GDscript and needs lower level code. One also needs to keep the height data in CPU memory for this. There may be artefacts because of slight differences in how the surface is calculated on the GPU (visible) and CPU (physics) end.

For "infinite" terrain you need swapping mechanisms anyway, on both ends. This is best done during loading or generation of the height texture. Communicating data back from a shader to the CPU is rather involved.

Alternatively, one could transfer all physics computation to the GPU where the data is available. Some physics APIs allow the acceleration with the GPU, but Godot now brings its own physics on the CPU.

Surpass

Answer 5

The problem with GPU physics is that they basically have to stay on the GPU. There is still non-trivial cost to getting the data back to the CPU.

For example, with PhysX, you could typically only accelerate non-gameplay related effects. For example Batman's cape, hair, water and smoke, bullet debris, etc. since it did not need to be accessible on the CPU.

Goosander

Answer 6

Yeah that's what I hoped: That once the shader generates a texture (including vertex offset) it can send that texture back to the CPU so GDScript can use get_pixel() on it like any other image. But I understand that isn't really possible... even if it was other limitations would remain. CPU with threading remains best for this, once you finally get threads to stop crashing every minute.

Hodometer

Answer 7

Depends how you do it. I have the height data as single channel 16bit textures on the CPU and upload them as 16 bit floats to the GPU for rendering. Float because GLSL doesn't allow type casts. I keep a copy as 16bit data on the CPU for frustum checks and to perform ray casts against, to prevent the camera from looking at the radish from below. So no to and fro between GPU and CPU is necessary.

This runs even with 600 fps on my new 5600G full hd, but it's C++, single threaded.

Surpass

Answer 8

I'd have to look at what's possible with Godot. I know like with Vulkan you can have a shared texture (CPU and GPU) but it removes the GPU-specific optimizations so it is slightly slower.

I don't know all that much about OpenGL or what is exposed in Godot. I know you can read textures using get_pixel(), but you have to lock it first and it's extremely slow for an HD texture, so would cause stutter at run-time.

But I guess if you had chunks of terrain that loaded in and out, and you processed it on a separate thread, it might be possible to do it in the background without stutter. But I haven't tried this, I'm not sure.

Goosander

Answer 9

Yeah, this is one method in GLSL as well, and performance is horribly because the texture has to be kept in CPU accessible memory, and some mapping and synchronisation must take place to prevent garbage.

Instead, jut keep a copy of the data in CPU mem, no hassle at all, as small as it can get, and it works.

Surpass

Answer 10

You could do that, but then the mesh could not be changed on the GPU, for example for dynamic deformation. If all the relevant data can be on the CPU the whole time, then it seems smarter to do this with MeshDataTool and GDScript since there is not much advantage to having it in the GPU if it's static.

Goosander