LODding Terrain for 3D in Godot
Asked Answered
I

33

0

Hi,

let's see, can you follow this tutorial until the hello-triangle (and further on if you wish ofc.) ?

You would additionally have to install the other mentioned dependencies for my cdlod, without the loader glad. Then you're set, at least you could forward me the MS compiler's error messages.

But I cannot help you debugging your environment, should problems arise there. So, you either can compile and run the triangle, or you can not, I am as sorry as possible :-)

Illboding answered 4/9, 2022 at 15:58 Comment(0)
S
0

Illboding It might be a good tutorial... but it's obviously not for a quick check… you have to dive in, figure it out and install the missing stuff… it's time consuming. Is there a difference — Win 7 or Win 11 — which one is needed? I'll try to make time in the next few days.

Samekh answered 4/9, 2022 at 16:59 Comment(0)
I
0

I certainly get it, no worries. I mean, we'd obviously need some deeper C++ understanding here that I did not mention. Nevertheless, if you like to dive in, there you go. Have fun :-)

Illboding answered 4/9, 2022 at 17:32 Comment(0)
S
0

Here is an open source C++ project and they have a "Procedural terrain generator". Might be interesting.

Samekh answered 5/9, 2022 at 8:37 Comment(0)
I
0

Dana

The Wicked Engine is a very cool project with lots of high tech, but still over my head 🙄

Illboding answered 5/9, 2022 at 9:0 Comment(0)
S
0

Illboding To the greatest regret, I can't help in the near future. It turns out that you have to register with M$ to work with Visual Studio… If you can't find a helper, I may do it later, perhaps in time, but for now I'm not morally ready… I'm extremely sorry.

Samekh answered 5/9, 2022 at 19:7 Comment(0)
I
0

No worries !

Illboding answered 5/9, 2022 at 19:41 Comment(0)
E
0

Illboding I can do that (VS2017, 2019, or 2022), but not this week, as I'm away at a conference. But I can do it next week.

Ephod answered 6/9, 2022 at 17:39 Comment(0)
I
0

Refactoring. Pls. stand by :-/

Am currently simplifying the conversions between raster space, the space represented by the raster of the spacial data structure which are basically integers for the positions in heightmap textures, and world space, which is the usual floating point stuff one deals with in any 3d application.

Before there was the possibility to extend a terrain in all directions, but that lead to conversion problems I could not really wrap my brain around. Si si, somos aficionados ...

So, in future I will start any quad tree's raster at 0,0,0 and extend it out to max, which is limited to power of 2 sizes. That makes things considerably easier, also when I think of presenting everything to people who probably just want to slab in a bunch of textures and expect it to just work.

Since I also have a new home to build, this may all take a bit longer.

Lesson learnt: Do not say it will be done until it is done.

Illboding answered 13/9, 2022 at 10:47 Comment(0)
I
0

The last overworking got me one step forward. That's from the rim of the canyon where I stood :-)

Never mind the flat shading. Shown is one terrain tile made from a 16bit heightmap 4096*4096 pixels, but it works well up to a tile size of the maximum the graphics card can digest. My test data is SRTM (Shuttle Radar Topographic Mission) data, shown is a part of the Himalaya, distance between two posts is about 90m. Purposefully exaggerated to better show the boxes that cover the terrain for view frustum selection.

The white lines are bounding box of the whole tile, the light blue ones are all the boxes of each node of the spatial data structure, the coloured ones are the per frame selected lod levels as seen from camera position.

Sending two 0s away gives us:

with the bounding boxes of the lod levels.

A close-up of a valley gives an impression of what the terrain lod is planned for (no planetary curvature yet, or any sophisticated shading, that's future music for now):

Illboding answered 17/9, 2022 at 18:49 Comment(0)
I
0

So, the base algorithm is finally working.

https://github.com/BaronVerde/cdlod

Tagging Pyatt , whenever you like.

A 4k heightmap texture comes with it. Ask me if you want a 16k texture, or search SRTM 90m data download, select the region you want, choose ARC ASCII as format, and see my other rep srtm-converter for how to generate .png tiles from the SRTM ASCII data. Also, try noise textures you may have somewhere. Eventually tweak the HEIGHT_FACTOR in settings.h.

Loading of a large heightmaps takes a few seconds, though that's going to change. Please mark the b* and krams directories under src as exclude from compile or you get multiple definition errors. These are just backups from working states.

It is only a single heightmap texture for now, and only flat vertex shading. No cosmetics. If you use an own heightmap, make sure it is 16bit-grayscale png of power-of-2, edit RASTER_MIN and MAX values in src/applications/cdlod/settings.h, and add a .bb text-file next to the heightmap texture. Look into src/renderer/renderer.cpp and change the filename there. Don't add .png. Attention, though there is a vector defined there, only one texture is loaded.

Next steps:
Comb the code. There are quite a few implicit conversions I must get rid off. - done -
Texture compression. - scrubbed, using half floats and uint16_t -
Parametrize and correctly calculate raster-to-world sizes and distances. My poor brain ...
Abstract texture types and bit depths and quite generally heightmap sources.
Build a data structure for heightmaps, and load/unload them asynchronously.
Render relative to eye. That means an own terrain camera.

Then: try to get it into the Godot scene tree.

Then proper texturing, rendering, evtl. cascaded shadow maps that fit the lod levels and ranges, collisions, placing of game objects.

Need a beer.

Illboding answered 28/9, 2022 at 21:55 Comment(0)
I
0

Effect of a resolution multiplier that equally divides the space between two heightmap posts and thus "fakes" a higher resolution than there actually is. It is just a linear interpolation. Considerably improves visual experience (look at the detail in the distance), raises number of triangles rendered, and adversely affects performance :-/

4*resolution multiplier, 11 million triangles per frame, frame rate drop from around 5000/s to 1000/s compared to the other screenshots which are from the same heightmap.

Illboding answered 29/9, 2022 at 10:40 Comment(0)
M
0

Illboding Considerably improves visual experience ... and adversely affects performance :-/

Look, sometimes it's worth it, ok? 😉

1ms vs 0.2ms is not a big deal.

Marinetti answered 29/9, 2022 at 12:37 Comment(0)
I
0

Certainly. Specifically comes in handy when using procedurally generated maps from noise or fractals. And there are other methods to control the density in a distance. Otoh, proper shading isn't yet done, nor shadows. So, it will not stay that comfortable.

But of course, having a higher resolution heightmap right from the start would be best. The resolution in the example is also not exactly game-friendly, it is around 90m between posts. A small thorpe or village would just be a speck from that perspective.

Have also experimented with different methods to calculate the normals, which is done on the fly in the vertex shader so I don't have to do normal maps for the terrain. Will offer several methods further down, to control better the "crispyness" of the image.

Illboding answered 29/9, 2022 at 13:42 Comment(0)
S
0

Huge work. Unfortunately, all the landscape generators do not take into account the principle of the formation of rivers. Rivers are not just irregularities filled with water. Until hydrodynamics in terrain formation is implemented, even the best generators will only be able to automatically create a lunar landscape. Or require a lot of manual revision.

Samekh answered 29/9, 2022 at 15:24 Comment(0)
I
0

Thanks :-) Yeah, my stuff it just a renderer for heightmap images that stem from elsewhere. I will do an interface that people can use to adapt their own heightmaps.

But there is quite some work on modelling of drainage systems. Look at this, for instance:
https://hal.archives-ouvertes.fr/hal-01339224/document

Update:
OpenGL built-in texture compression for heightmap (RGTC) is scrubbed. Too lossy.

Illboding answered 29/9, 2022 at 16:5 Comment(0)
S
0

Illboding But there is quite some work on modelling of drainage systems. Look at this, for instance:
https://hal.archives-ouvertes.fr/hal-01339224/document

This is excellent work! But as far as I understand, it remained at the level of the concept and did not get any practical questioning. And that's too sad.

Samekh answered 29/9, 2022 at 16:41 Comment(0)
I
0

They have an implementation, they write.

And there's more work out there. I believe Outerra has made something similar. And I have read papers in which they try to mimic plate tectonics. Don't have the time to gather the information again.

Bit I'd say the algorithm in the above paper could be implemented right away.

Illboding answered 29/9, 2022 at 16:46 Comment(0)
S
0

Illboding They have an implementation, they write.

It is all very academic. Until the implementation in game engines, apparently, very far away.

Bit I'd say the algorithm in the above paper could be implemented right away.

I definitely can't pull off such a job. 🙁

Samekh answered 29/9, 2022 at 16:58 Comment(0)
E
0

Illboding I'll try to hit this tomorrow. :-)

Ephod answered 30/9, 2022 at 2:47 Comment(0)
E
0

Illboding I don't see a makefile, or CMakeLists.txt, or similar to describe the source dependencies. Do I need to make one?

Ephod answered 30/9, 2022 at 2:53 Comment(0)
I
0

Yes. You can use whatever build system you like or your IDE supports (even Scons 😜 ). I just use the Eclipse-cdt built-in.

I set include paths to the src/ and extern/ subdirectories. All includes should be relative (sometimes Eclipse gets in the way and renames them). Refer to the readme.txt for other dependencies. Link against the dynamic loader and pthread. Well, you'll see :-)

I am not into the make family because they drop their files in every directory.

Illboding answered 30/9, 2022 at 7:51 Comment(0)
I
0

I have something to ponder.

The initial version (that btw.. compiled on Windows with only minor adaptations, thank you Pyatt) was based on a heighmap that is available in its entirety right from the beginning. I got away without any pre-processing of data, just load the texture image and start rendering.

I said I build a data structure of aabbs of given node_size that are used to select what needs to be rendered this frame. For that, I need the horizontal extent which is trivially known by the node's x/z position in the quadtree. And the minimum and maximum height, either thoroughly looked up from all the node's height values which is slow but accurate or estimated for instance from lookups of corner points and centre or any selection of values inside of the node. More formally: a node's aabb at position x, z in the raster of the quadtree has the coordinates

minimum = (x, minimum height of node's area, z)
maximum = (x+node_size-1, maximum height of node's area, z+node_size-1)

Savvy so far ? 🏴‍☠️

I did this to have optimal flexibility, as everything terrain renderer related could be recalculated on the fly.

These days seem to be over when I expand this to something that doesn't fit into memory any more. The quadtree is generated as a whole, but heightmap tiles will be loaded on demand. So he horizontal extent of a node is still there as it only depends on the quadtree, but the node's height values are not yet in memory.

I could

  • (a) just ignore them and use an aabb of magical height for the selection.
  • (b) pre-calculate minimum and maximum values.
  • (c) look them up each frame during node selection.
  • (d) insert them into the nodes' aabbs on tile loading.

(a) Not feasible. In steep terrain nodes will be missed.
(b) Possible. Needs a pre-calculation step that extracts minimum and maximum values when the heightmap images are generated. Or I would have to offer a tool. Disadvantage: I loose quite some flexibility as the node size must be known when a heightmap is generated. This means people should know what area and spatial extent a heightmap is meant to be for when they make one.
(c) Possible for small heightmaps, but quickly slows down the algorithm because of cache misses.
(d) Possible, but would probably lead to some popping and actually I wanted to avoid this type of artefact
But it would be slow as I'd have to traverse the tree to find each node and assign the minimum and maximum values. It'll also significantly slows down the loading process for a tile.

Maybe I have not thought of something.

I tend to (b), create a min-max-map and sacrifice flexibility. Creating such a map would be a nice finger exercise for an aspiring C++ newcomer. I'll help :-)

Illboding answered 2/10, 2022 at 17:32 Comment(0)
M
0

IMO, Either b) or d) or if you can afford to implement both, option e) "choose your poison", let the game designer choose which works best for them.

But if only one, keeping in mind that

Illboding actually I wanted to avoid this type of artefact

I'd also go for b)

Marinetti answered 3/10, 2022 at 5:56 Comment(0)
I
0

That would be possible to do both methods, it is not a big deal logic wise, just diligence.

Currently doing a small generator (diamond square) for terrain at higher resolution than I can do with real world data. I need some realistic values suitable for a game environment for testing.

Illboding answered 3/10, 2022 at 10:16 Comment(0)
S
0

Illboding Disadvantage: I loose quite some flexibility as the node size must be known when a heightmap is generated. This means people should know what area and spatial extent a heightmap is meant to be for when they make one.

I guess when the user creates a landscape, they assume what terrain they want — mountainous, hilly or plain.

Probably need to distinguish between generators and pseudo terrain generators.

Illboding I need some realistic values suitable for a game environment for testing.

I will show it with my own example. I need a terrain generator, first for a small village-farm, which could then grow into a city: a river or lake nearby, a forest of small hills.

For example for archaeologists, you need accurate reconstruction of terrain on the map — for the reconstruction of ancient settlements. But an element of randomness is desirable for the game.

Specifically, here is a desirable example of terrain:

This, the ancient Russian city — the Lord Great Novgorod. The last democratic state of Russ.

It is a swampy land, it is not always clear which rivers are natural and which are artificial. But to develop the city you need the ability to change the landscape: cut down trees, block the river in one place and dig canals in another. Fill in and tear down hills.

Samekh answered 3/10, 2022 at 11:26 Comment(0)
I
0

Dana Looks nice, I genuinely love archaeology.

I herewith lift you in the rank of chief terrain generation algorithms officer (CTGO). Your task is to search and find (and optionally implement) algorithms suitable for the purposes you want in a game :-)

A quick and dirty diamond square with fixed values is just 150 lines of code, including writing images to file, which is more than half of that ...

Illboding answered 3/10, 2022 at 12:19 Comment(0)
S
0

Illboding Your task is to search and find (and optionally implement) algorithms suitable for the purposes you want in a game

Greatest regrettably, creating the terrain is only part of the project. There's a lot to be realized: the changing of the seasons, the growth of vegetation, the social behavior of people… I think I'll stop there. Right now I'm trying to figure out what can be implemented fairly cheaply in the first phase to proof-of-concept, and what should be put aside for later.

Samekh answered 3/10, 2022 at 12:34 Comment(0)
E
0

Option (e): instead of a quadtree, use an octree, where each octant in a given LOD is the same size.

Ephod answered 3/10, 2022 at 14:1 Comment(0)
I
0

But that doesn't help me with the problem that the height data is not available at the time the spacial data structure is built.

(b) is the way for height maps that are already defined before the program starts. Needs some pre-processing, but is fast.
(d) must be done when height data is generated procedurally, like not stored at all. Performance needs to be monitored.

Illboding answered 3/10, 2022 at 14:31 Comment(0)
E
0

It does help. With an octree, you don't have care about the min/max height of the source terrain data, b/c each octant is always the same size. If the terrain data exceeds the bounds of the current octant (above or below), that portion of the terrain data is put into a sibling octant (i.e. the octant above or below the current octant). This way your LOD switching distances are constant for a given LOD.

Ephod answered 3/10, 2022 at 14:40 Comment(0)
I
0

I had to pause for almost a month, but this weekend managed to implement the independent scaling of linear flat worlds. This means, one passes in the heightmap und a constant metric distance between posts and the renderer takes care of the scaling.

Haven't yet finalized the double precision rendering, so for now there is jittering when close to terrain of "actual size", but I got a roadmap for that.

Next asynchronous loading of heightmap patches, henceforth named tiles.

Maybe I should start to learn how to do videos ...

Illboding answered 31/10, 2022 at 18:29 Comment(0)
M
0

Illboding Maybe I should start to learn how to do videos ...

OBS

Marinetti answered 31/10, 2022 at 22:5 Comment(0)
I
0

Well, I was thinking about a recorder for my own renderer, but I think that'll do for now :-)

I still believe, that if I integrate this into the Godot scene tree, it'll be a huge node that brings all with it that is needed for rendering the world, terrain, scatterings, sky, shadows. And visitable places in there may be other scenes.

Illboding answered 31/10, 2022 at 23:8 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.