OpenGL VBO within multiple threads

Question

I am developing a program in C++/OpenGL which draws terrain of the entire world. I have a database of altitude heights stored as tiles. Every time I start the program, a tile is loaded. Then as the person moves, another tile should load, this does not happen every frame, maybe once every 5 minutes.

I load the initial tile in the video card's memory:

glBindBufferARB(GL_ARRAY_BUFFER_ARB, VertexBuffer[idx]);
glBufferDataARB(GL_ARRAY_BUFFER_ARB, VBOsz * 3 * sizeof(float), tile_data, GL_STATIC_DRAW_ARB);

...There are normals, color and index buffers

And I draw them:

glBindBufferARB(GL_ARRAY_BUFFER_ARB, VertexBuffer[idx]);
glBufferDataARB(GL_ARRAY_BUFFER_ARB, VBOsz * 3 * sizeof(float), tile_data, GL_STATIC_DRAW_ARB);


glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_COLOR_ARRAY);
glEnableClientState(GL_NORMAL_ARRAY);

glBindBufferARB(GL_ARRAY_BUFFER_ARB, VertexBuffer[idx]);
glVertexPointer(3, GL_FLOAT, 0, BUFFER_OFFSET(0));

...

glBindBufferARB(GL_ELEMENT_ARRAY_BUFFER_ARB, IndexBuffer[idx]);
glDrawElements(GL_TRIANGLES, IndexBuffersz, GL_UNSIGNED_INT, BUFFER_OFFSET(0));

glDisableClientState(GL_NORMAL_ARRAY);
glDisableClientState(GL_COLOR_ARRAY);
glDisableClientState(GL_VERTEX_ARRAY);

Since I want the program to be as smooth as possible, I can't calculate the vertex+color+normal+ other textures the same frame, as it takes about 20 seconds to create a tile.

So I decided to make a loader thread that would check for when a new tile needs to be loaded and then load it. When it would be all done, it should just swap the VBO(hence the [idx].

So for a loader thread, I know that I need a second OpenGL context, I created one and I share the lists between them. The idea is working, but in the loader thread, when I send the new VBO data, I need this function: wglMakeCurrent

Only when all the data has been loaded, I can set the context to the rendering thread(main program's thread). This causes nothing to be drawn for that amount of time, which makes the program useless.

Do you have any ideas on a solution? Do I need to change the concept?

I am using OpenGL 2.1. Will upgrading to OpenGL 3 solve the problem?

Cottingham

Please stop using the ARB-extension form of buffer objects. They've been core functionality for almost a decade; there's no excuse anymore. – Danikadanila

Answer 1

I have already an answer for this kind of tasks.

In few words, you create two sharing contextes. Then, as suggested by Damon, make the contextes current on their own thread, only once at the beginning of the thread execution. The two contextes will be current at the same time on different threads (one thread, one context).

Then, the secondary thread won't be used for rendering, but for loading resources (I mean, actually loading terrain data, textures... and create a corresponding OpenGL object on each data, such as textures and buffer objects). This happens while the main context is drawing.

Essentially, you don't need to worry about bringing a pointer around the application and blocking the rendering thread for uploading data; but at the cost of creating another context. Let the driver to synchronize context states: if it can perform that operations smoothly your application will will benefit of it; at least, you code will be cleaner.

Other contributions of mine on the subject:

• Resource design : read about resource sharing
• Sharing between contextes with different versions

Phytosociology

Thank you, this is exactly how I have it set up: a secondary thread that does the work, with the contexts shared. For some applications, blocking the rendering is not an option. – Cottingham

Answer 2

This is really not that complicated.

You just make two buffer objects: one that you're using for current rendering and one that you will be using in the future. When the unused buffer is full of data, the rendering thread switches to rendering from that one. The previously used buffer becomes the unused one.

You can upload the data in one of two ways. One way is for your data creation thread to create an array of data that the rendering thread will upload to the buffer object with glBufferData. Obviously this will require some synchronization, but it's sync code that you need: your render thread must ultimately be informed when data is ready so that it can render with the new data.

The other way is for your rendering thread to be told that data needs to start being generated. At which point, it will map the unused buffer object and pass the mapped pointer to the data creation thread. That thread will generate data directly into that mapped pointer. When it finishes, it informs the render thread, which will unmap the buffer and then render with the data.

Neither method requires multiple contexts or threading through OpenGL code.

Note that performance will be best served by not making the buffers bigger and smaller. Pick a size and stick with it; you don't want to resize buffers with glBufferData.

Danikadanila

I tried both methods, but glBufferData takes at least 200 milliseconds and mapping/unmapping also takes 100 milliseconds. In my application I can not have this because it would stutter. The buffer size would not change. – Cottingham

@Tibi: Are you lying to the OpenGL implementation, by using STATIC_DRAW when your data isn't actually static? – Danikadanila

I do not modify the VBO data, I am using double-buffering. So I guess that STATIC_DRAW is fine, right? – Cottingham

@Tibi: No. STATIC_DRAW means that you upload data once. After this upload, the buffer is fixed and unchanging. By calling glBufferData on it again, you are violating this constraint. – Danikadanila

Let me explain this better, we have buffer 0, which is initially used and buffer 1. When the code decides that it needs to load other data, it loads it into buffer 1 and then the rendering renders from that buffer. Buffer 0 can be deleted and the next time the loading thread calculates a redraw, it recreates the buffer. – Cottingham

@Tibi: Deleting and recreating objects is not something you should be doing in the middle of a rendering frame. Both buffers need to be kept around. – Danikadanila

So it would be better to create them with DYNAMIC_DRAW and use glBufferData to update accordingly? – Cottingham

Answer 3

You only need to call wglMakeCurrent exactly once in each thread. This works reliably, it is what I'm doing (though with OpenGL 3.3). This marks one context belonging to one thread. It stays that way until you tell OpenGL differently, so do it once at the beginning and forget (in fact, you don't need to call it at all if you create contexts in their respective threads using them, but do it anyway just to be 100% safe, also I prefer creating all contexts before starting up, it's not as messy...).

You need not worry about the function pointer, by the way, just use the same one you've used in the render thread (assuming you've properly initialized it there).
Technically, it is invalid to use a function pointer from another context. However, WGL kindly guarantees (hidden in the small print!) that function pointers are identical for all contexts having the same pixel format. Thus, you're good to go.

An alternative that works with a single context is to glMapBuffer in the render thread and pass the pointer to the worker thread. Then, upon completion (signalling a semaphore, for example), glUnmapBuffer, again in the render thread.
Some people prefer this, as it does not involve context juggling and presumably works better on some old buggy drivers. I don't like it because of the extra synchronization needed. It's a matter of taste, same effect.

Headword

Thank you for your help. Adding wglMakeCurrent as the beginning of the rendering thread makes it work. However, on my first try I had a computer freeze. I am not sure of the exact line, but it happened when calling wglMakeCurrent in the loader thread, or right after that: glBindBuffer and glBufferData. Now it seems to be working fine. Is it bad coding to be blamed for this? – Cottingham

@Tibi: It sounds more like a threading mishap. What steps have you taken to ensure that you're not using the same object on both threads at the same time? – Danikadanila

@Nicol: In the loader thread I have a variable, only after it finishes doing its job, I set the variable to true. Then the rendering thread does the actual "flip" of the buffers. Once the data is loaded, the loading thread does not do any OpenGL work. – Cottingham

@Tibi: If that variable is not a proper mutex or atomic value or something, then this will fail. Threading is something you need to fully understand before you start trying to thread graphics work. – Danikadanila