How to use VAOs with instancing in Qt 5

Question

I'm trying to wrap my head around how to use VAOs appropriately for instanced rendering (specifically in Qt 5.2, using OpenGL 3.3). My understanding is that VAOs save the state of the VBOs and associated attributes so that you don't need to worry about binding and enabling everything at drawing time, you just bind the VAO. But with instancing, you often have multiple VBOs. How do you get around needing to bind them all? Or do I just need to use a single VBO for both my per vertex data and my per instance data?

I've been looking at a couple tutorials, for example: http://ogldev.atspace.co.uk/www/tutorial33/tutorial33.html

It looks to me like what he does is use a VAO for his per vertex data and NOT for his per instance data. I've tried doing the same thing with my Qt-based code, and it's not working for me (probably because I don't entirely understand how that works... shouldn't his instance data still need to be bound when drawing happens?)

Some dummy code... this is a bit silly, I'm just drawing a single instance of two triangles, with a perspective matrix as a per instance attribute.

glwindow.cpp:

#include "glwindow.h"

#include <QColor>
#include <QMatrix4x4>
#include <QVector>
#include <QVector3D>
#include <QVector4D>

#include <QDebug>


GLWindow::GLWindow(QWindow *parent) 
  : QWindow(parent)
  , _vbo(QOpenGLBuffer::VertexBuffer)
  , _matbo(QOpenGLBuffer::VertexBuffer)
  , _context(0)
{
  setSurfaceType(QWindow::OpenGLSurface);
}

GLWindow::~GLWindow()
{}

void GLWindow::initGL()
{  
  setupShaders();
  _program->bind();
  _positionAttr = _program->attributeLocation("position");
  _colourAttr = _program->attributeLocation("colour");
  _matrixAttr = _program->attributeLocation("matrix");

  QVector<QVector3D> triangles;
  triangles << QVector3D(-0.5, 0.5, 1) << QVector3D(-0.5, -0.5, 1) << QVector3D(0.5, -0.5, 1);
  triangles << QVector3D(0.5, 0.5, 0.5) << QVector3D(-0.5, -0.5, 0.5) << QVector3D(0.5, -0.5, 0.5);

  QVector<QVector3D> colours;
  colours << QVector3D(1, 0, 0) << QVector3D(0, 1, 0) << QVector3D(0, 0, 1);
  colours << QVector3D(1, 1, 1) << QVector3D(1, 1, 1) << QVector3D(1, 1, 1);

  _vao.create();
  _vao.bind();

  _vbo.create();
  _vbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
  _vbo.bind();

  size_t positionSize = triangles.size() * sizeof(QVector3D);
  size_t colourSize = colours.size() * sizeof(QVector3D);
  _vbo.allocate(positionSize + colourSize);
  _vbo.bind();
  _vbo.write(0, triangles.constData(), positionSize);
  _vbo.write(positionSize, colours.constData(), colourSize);
  _colourOffset = positionSize;

  _program->setAttributeBuffer(_positionAttr, GL_FLOAT, 0, 3, 0);
  _program->setAttributeBuffer(_colourAttr, GL_FLOAT, _colourOffset, 3, 0);

  _program->enableAttributeArray(_positionAttr);  
  _program->enableAttributeArray(_colourAttr);

  _vao.release();

  _matbo.create();
  _matbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
  _matbo.bind();

  _matbo.allocate(4 * sizeof(QVector4D));
  _program->setAttributeBuffer(_matrixAttr, GL_FLOAT, 0, 4, 4 * sizeof(QVector4D));
  _program->enableAttributeArray(_matrixAttr);

  _func330->glVertexAttribDivisor(_matrixAttr, 1);
  _matbo.release();

  _program->release();
  resizeGL(width(), height());
}

void GLWindow::resizeGL(int w, int h)
{
  glViewport(0, 0, w, h);
}

void GLWindow::paintGL()
{
  if (! _context) // not yet initialized
    return;

  _context->makeCurrent(this);
  QColor background(Qt::black);

  glClearColor(background.redF(), background.greenF(), background.blueF(), 1.0f);
  glClearDepth(1.0f);
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
  QMatrix4x4 matrix;
  matrix.perspective(60, 4.0/3.0, 0.1, 100.0);
  matrix.translate(0, 0, -2);

  _program->bind();

  _matbo.bind();
  _matbo.write(0, matrix.constData(), 4 * sizeof(QVector4D));

  _vao.bind();

  glEnable(GL_DEPTH_TEST);
  _func330->glDrawArraysInstanced(GL_TRIANGLES, 0, 6, 1);

  _vao.release();

  _program->release();

  _context->swapBuffers(this);
  _context->doneCurrent();

}

void GLWindow::setupShaders()
{

  QString vShaderSrc("#version 330\n"
    "layout(location = 0) in vec4 position;\n"
    "layout(location = 1) in vec4 colour;\n"
    "layout(location = 2) in mat4 matrix;\n"
    "smooth out vec4 col;\n"
    "void main() {\n"
    "   col = colour;\n"
    "   gl_Position = matrix * position;\n"
    "}\n");

  QString fShaderSrc("#version 330\n"
    "smooth in vec4 col;\n"
    "void main() {\n"
    "   gl_FragColor = col;\n"
    "}\n");


  _program = new QOpenGLShaderProgram(this);
  _program->addShaderFromSourceCode(QOpenGLShader::Vertex, vShaderSrc);
  _program->addShaderFromSourceCode(QOpenGLShader::Fragment, fShaderSrc);
  _program->link();


}

void GLWindow::exposeEvent(QExposeEvent *event)
{
  Q_UNUSED(event);

  if (isExposed())
  {
    if (! _context)
    {
      _context = new QOpenGLContext(this);
      QSurfaceFormat format(requestedFormat());
      format.setVersion(3,3);
      format.setDepthBufferSize(24);

      _context->setFormat(format);
      _context->create();

      _context->makeCurrent(this);
      initializeOpenGLFunctions();

      _func330 = _context->versionFunctions<QOpenGLFunctions_3_3_Core>();
      if (_func330)
        _func330->initializeOpenGLFunctions();
      else
      {
        qWarning() << "Could not obtain required OpenGL context version";
        exit(1);
      }

      initGL();
    }

    paintGL();
  }
}

glwindow.h:

#ifndef GL_WINDOW_H
#define GL_WINDOW_H

#include <QExposeEvent>
#include <QSurfaceFormat>
#include <QWindow>

#include <QOpenGLBuffer>
#include <QOpenGLContext>
#include <QOpenGLFunctions>
#include <QOpenGLFunctions_3_3_Core>
#include <QOpenGLShaderProgram>
#include <QOpenGLVertexArrayObject>


class GLWindow : public QWindow, protected QOpenGLFunctions
{
  Q_OBJECT
public:
  GLWindow(QWindow * = 0);
  virtual ~GLWindow();

  void initGL();
  void paintGL();
  void resizeGL(int, int);

protected:
  virtual void exposeEvent(QExposeEvent *);


private:

  void setupShaders();

  QOpenGLBuffer _vbo;
  QOpenGLBuffer _matbo;
  QOpenGLContext *_context;
  QOpenGLShaderProgram *_program;
  QOpenGLVertexArrayObject _vao;
  QOpenGLFunctions_3_3_Core *_func330;

  GLuint _positionAttr;
  GLuint _colourAttr;
  GLuint _matrixAttr;

  size_t _colourOffset;

} ; 

#endif

glbuffertest.cpp:

#include <QGuiApplication>
#include <QSurfaceFormat>

#include "glwindow.h"



int main(int argc, char **argv)
{

  QGuiApplication app(argc, argv);

  GLWindow window;  
  window.resize(400, 400);
  window.show();

  return app.exec();

}

glbuffertest.pro:

######################################################################
# Automatically generated by qmake (3.0) Fri May 16 09:49:41 2014
######################################################################

TEMPLATE = app
TARGET = glbuffertest
INCLUDEPATH += .

CONFIG += qt debug

# Input
SOURCES += glbuffertest.cpp glwindow.cpp
HEADERS += glwindow.h

UPDATE: I've tried getting rid of my _matbo buffer and instead putting the matrix data into the same VBO as the position and colour attributes, but it's not working for me. My initGL function now looks like:

void GLWindow::initGL()
{  
  setupShaders();
  _program->bind();
  _positionAttr = _program->attributeLocation("position");
  _colourAttr = _program->attributeLocation("colour");
  _matrixAttr = _program->attributeLocation("matrix");

  QVector<QVector3D> triangles;
  triangles << QVector3D(-0.5, 0.5, 1) << QVector3D(-0.5, -0.5, 1) << QVector3D(0.5, -0.5, 1);
  triangles << QVector3D(0.5, 0.5, 0.5) << QVector3D(-0.5, -0.5, 0.5) << QVector3D(0.5, -0.5, 0.5);

  QVector<QVector3D> colours;
  colours << QVector3D(1, 0, 0) << QVector3D(0, 1, 0) << QVector3D(0, 0, 1);
  colours << QVector3D(1, 1, 1) << QVector3D(1, 1, 1) << QVector3D(1, 1, 1);

  _vao.create();
  _vao.bind();

  _vbo.create();
  _vbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
  _vbo.bind();

  size_t positionSize = triangles.size() * sizeof(QVector3D);
  size_t colourSize = colours.size() * sizeof(QVector3D);
  size_t matrixSize = 4 * sizeof(QVector4D);
  _vbo.allocate(positionSize + colourSize + matrixSize);
  _vbo.bind();
  _vbo.write(0, triangles.constData(), positionSize);
  _vbo.write(positionSize, colours.constData(), colourSize);


  _colourOffset = positionSize;
  _matrixOffset = positionSize + colourSize;

  _program->setAttributeBuffer(_positionAttr, GL_FLOAT, 0, 3, 0);
  _program->setAttributeBuffer(_colourAttr, GL_FLOAT, _colourOffset, 3, 0);
  _program->setAttributeBuffer(_matrixAttr, GL_FLOAT, _matrixOffset, 4, 4 * sizeof(QVector4D));

  _program->enableAttributeArray(_positionAttr);  
  _program->enableAttributeArray(_colourAttr);
  _program->enableAttributeArray(_matrixAttr);
  _func330->glVertexAttribDivisor(_matrixAttr, 1);

  _vao.release();

  _program->release();
  resizeGL(width(), height());
}

and paintGL:

void GLWindow::paintGL()
{
  if (! _context) // not yet initialized
    return;

  _context->makeCurrent(this);
  QColor background(Qt::black);

  glClearColor(background.redF(), background.greenF(), background.blueF(), 1.0f);
  glClearDepth(1.0f);
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
  QMatrix4x4 matrix;
  matrix.perspective(60, 4.0/3.0, 0.1, 100.0);
  matrix.translate(0, 0, -2);


  _program->bind();

  _vao.bind();
  _vbo.write(_matrixOffset, matrix.constData(), 4 * sizeof(QVector4D));

  /* I tried replacing the three preceding lines with the following, without success: */

  /*
  _vao.bind();
  _vbo.bind();
  _vbo.write(_matrixOffset, matrix.constData(), 4 * sizeof(QVector4D));


  _program->bind();
  _program->enableAttributeArray(_matrixAttr);
  _func330->glVertexAttribDivisor(_matrixAttr, 1); */

  glEnable(GL_DEPTH_TEST);
  _func330->glDrawArraysInstanced(GL_TRIANGLES, 0, 6, 1);

  _vao.release();

  _program->release();

  _context->swapBuffers(this);
  _context->doneCurrent();

}

So it seems my instancing problems are bigger than just having the wrong buffer bound at the wrong time. What else am I doing wrong?

Upbuild

Answer 1

I've got it. The main problems were that:

1. I had to loop through all four columns of my mat4 attribute, setting and enabling each of them, and calling glVertexAttribDivisor() on each.
2. I had completely messed up the call to QOpenGLShaderProgram::setAttributeBuffer() for my mat4 attribute.

Essentially, you have to treat a mat4 as four separate vec4 attributes (one for each column). This doesn't affect how you copy QMatrix4x4 data to a QOpenGLBuffer object in the slightest, just how you tell the shader program to deal with the data. This is well described in both the tutorial I linked to in my original question and in The OpenGL Programming Guide's instancing tutorial, I just didn't get it. So, going back to the first attempt at glwindow.cpp above, I've changed very little and things now work:

#include "glwindow.h"

#include <QColor>
#include <QMatrix4x4>
#include <QVector>
#include <QVector3D>
#include <QVector4D>

#include <QDebug>


GLWindow::GLWindow(QWindow *parent) 
  : QWindow(parent)
  , _vbo(QOpenGLBuffer::VertexBuffer)
  , _matbo(QOpenGLBuffer::VertexBuffer)
  , _context(0)
{
  setSurfaceType(QWindow::OpenGLSurface);
}

GLWindow::~GLWindow()
{}

void GLWindow::initGL()
{  
  setupShaders();
  _program->bind();
  _positionAttr = _program->attributeLocation("position");
  _colourAttr = _program->attributeLocation("colour");
  _matrixAttr = _program->attributeLocation("matrix");

  QVector<QVector3D> triangles;
  triangles << QVector3D(-0.5, 0.5, 1) << QVector3D(-0.5, -0.5, 1) << QVector3D(0.5, -0.5, 1);
  triangles << QVector3D(0.5, 0.5, 0.5) << QVector3D(-0.5, -0.5, 0.5) << QVector3D(0.5, -0.5, 0.5);

  QVector<QVector3D> colours;
  colours << QVector3D(1, 0, 0) << QVector3D(0, 1, 0) << QVector3D(0, 0, 1);
  colours << QVector3D(1, 1, 1) << QVector3D(1, 1, 1) << QVector3D(1, 1, 1);

  _vao.create();
  _vao.bind();

  _vbo.create();
  _vbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
  _vbo.bind();

  size_t positionSize = triangles.size() * sizeof(QVector3D);
  size_t colourSize = colours.size() * sizeof(QVector3D);
  _vbo.allocate(positionSize + colourSize);
  _vbo.bind();
  _vbo.write(0, triangles.constData(), positionSize);
  _vbo.write(positionSize, colours.constData(), colourSize);
  _colourOffset = positionSize;

  _program->setAttributeBuffer(_positionAttr, GL_FLOAT, 0, 3, 0);
  _program->setAttributeBuffer(_colourAttr, GL_FLOAT, _colourOffset, 3, 0);

  _program->enableAttributeArray(_positionAttr);  
  _program->enableAttributeArray(_colourAttr);


  _matbo.create();
  _matbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
  _matbo.bind();

  _matbo.allocate(4 * sizeof(QVector4D));

  // This is completely wrong
  /*_program->setAttributeBuffer(_matrixAttr, GL_FLOAT, 0, 4, 4 * sizeof(QVector4D));
  _program->enableAttributeArray(_matrixAttr);

  _func330->glVertexAttribDivisor(_matrixAttr, 1);
  */      

  // The right way to set up a mat4 attribute for instancing
  for (unsigned i = 0; i < 4; i++)
  {
    _program->setAttributeBuffer(_matrixAttr + i, GL_FLOAT, i * sizeof(QVector4D), 4, 4 * sizeof(QVector4D));
    _program->enableAttributeArray(_matrixAttr + i);
    _func330->glVertexAttribDivisor(_matrixAttr + i, 1);
  }

  _matbo.release();
  _vao.release();

  _program->release();
  resizeGL(width(), height());
}

void GLWindow::resizeGL(int w, int h)
{
  glViewport(0, 0, w, h);
}

void GLWindow::paintGL()
{
  if (! _context) // not yet initialized
    return;

  _context->makeCurrent(this);
  QColor background(Qt::black);

  glClearColor(background.redF(), background.greenF(), background.blueF(), 1.0f);
  glClearDepth(1.0f);
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
  QMatrix4x4 matrix;
  matrix.perspective(60, 4.0/3.0, 0.1, 100.0);
  matrix.translate(0, 0, -2);

  _program->bind();
  _vao.bind();

  _matbo.bind();
  _matbo.write(0, matrix.constData(), 4 * sizeof(QVector4D));


  glEnable(GL_DEPTH_TEST);
  _func330->glDrawArraysInstanced(GL_TRIANGLES, 0, 6, 1);

  _vao.release();

  _program->release();

  _context->swapBuffers(this);
  _context->doneCurrent();

}

void GLWindow::setupShaders()
{

  QString vShaderSrc("#version 330\n"
    "layout(location = 0) in vec4 position;\n"
    "layout(location = 1) in vec4 colour;\n"
    "layout(location = 2) in mat4 matrix;\n"
    "smooth out vec4 col;\n"
    "void main() {\n"
    "   col = colour;\n"
    "   gl_Position = matrix * position;\n"
    "}\n");

  QString fShaderSrc("#version 330\n"
    "smooth in vec4 col;\n"
    "void main() {\n"
    "   gl_FragColor = col;\n"
    "}\n");


  _program = new QOpenGLShaderProgram(this);
  _program->addShaderFromSourceCode(QOpenGLShader::Vertex, vShaderSrc);
  _program->addShaderFromSourceCode(QOpenGLShader::Fragment, fShaderSrc);
  _program->link();


}

void GLWindow::exposeEvent(QExposeEvent *event)
{
  Q_UNUSED(event);

  if (isExposed())
  {
    if (! _context)
    {
      _context = new QOpenGLContext(this);
      QSurfaceFormat format(requestedFormat());
      format.setVersion(3,3);
      format.setDepthBufferSize(24);

      _context->setFormat(format);
      _context->create();

      _context->makeCurrent(this);
      initializeOpenGLFunctions();

      _func330 = _context->versionFunctions<QOpenGLFunctions_3_3_Core>();
      if (_func330)
        _func330->initializeOpenGLFunctions();
      else
      {
        qWarning() << "Could not obtain required OpenGL context version";
        exit(1);
      }

      initGL();
    }

    paintGL();
  }
}

Note that I also moved the binding of _matbo and setting up of the mat4 attribute so that it's all done before releasing the VAO. I was initially very confused over how many VBOs were allowed and when they needed to be bound. There's no problem having multiple VBOs inside a single VAO, it's just that the right one needs to be bound to be written to, and the right one needs to be bound before calling QOpenGLShaderProgram::setAttributeBuffer(). It doesn't matter which buffer is bound when glDraw*() is called (I trust someone will comment if I'm wrong about that).

Upbuild

Answer 2

I think you must create one VBO for positions and one VBO for colors (or use interleaved data with a stride). VAO allows you to use multiple VBO, one per attribute.

vao.create();
vao.bind();
// prepare your shader program
// ...
// prepare your VBOs : one VBO for pos, one VBO for colors, one for normals,...
// example for position
vertexPositionBuffer.create();
vertexPositionBuffer.setUsagePattern(QOpenGLBuffer::StaticDraw); 
vertexPositionBuffer.bind();
// if your store the points using QVector<QVector3D>
vertexPositionBuffer.allocate(vertices.constData(), vertices.size() * sizeof(QVector3D));
vertexPositionBuffer.release();
// do the same for colors or other attributes
// ...
// after all buffers are created
shaderProgram.bind();
// Bind the position buffer
vertexPositionBuffer.bind();
shaderProgram.enableAttributeArray("vertexPosition");
shaderProgram.setAttributeBuffer("vertexPosition", GL_FLOAT, 0, 3);
vertexPositionBuffer.release();
// do the same for all other buffers
// ...
shaderProgram.release();
// release vao 
vao.release();

and in your paintGL function:

// update your matrices
// bind your shader program
// set you uniform variables
// then
vao.bind();
glDrawArrays(GL_TRIANGLES, 0, vertices.size());
vao.release();
// release your shader program

Muley

Because I'm using instancing, some attributes also need to be updated at draw time, so I need to bind the appropriate buffers then. Sharing a buffer for multiple attributes isn't a problem with the offset method I used with my colours and positions attributes, but the VBO they're in doesn't seem to get bound when I bind my VAO again (at draw time, I mean). – Upbuild

1) You must bind the VBO before writing int it. 2) I dont think it's safe to write in the VBO inside your paintGL function after binding your program 3) You don't need to bind the VAO before writing in the VBO. 4) You should use DynamicDraw – Muley

2) It SHOULD be OK to write to a VBO inside the paintGL function... this is what's done with that instancing tutorial I linked to. It's necessary to use instancing, in fact. But only the currently bound VBO can be written to (I think... I can elaborate on what happens if you try to write to another one). 3) In fact, I'm NOT binding the VAO before writing to _matbo in the first code I posted. 4) I'm not sure how DynamicDraw might help :( – Upbuild

in paintGL _vbo.write without binding it? read QT doc : you must bin _vbo before writing in it. I don't understand what you realy want to do :your matrix seem to be constant so why not a uniform ? if you want to use instances your must provide attributes for each instance of your object (ex : one transformation matrix for each instance) : vbos to store arrays of attributes indexed by instanceId. – Muley

This is a dummy example. I'm just using a single matrix here to see if I can get anything to draw at all using a matrix as an attribute with the glDrawArraysInstanced function. In the actual software I'm working on, I need to draw a sphere hundreds of times in different sizes and different positions with different colours. As for _vbo in my dummy code, I'm not writing to it in my paintGL() function, and I AM binding it before writing to it in the initGL() function. – Upbuild

Don't show a dummy (wrong) code : show the code. Again the code you show here CAN'T work : you must bind _vbo before _vbo.write and release it after. change that, try again and come back here if that doesn't work. – Muley

My software is over 40,000 lines of code, I think posting it would be a tad excessive :) I'm quite happy to get this example working then work from there. In the code I have here, I DO bind the _vbo before writing to it. – Upbuild

No you don't: in paintGL you do _vao.bind (the vertex array, not the buffer) and just after _vbo.write. – Muley

Gotcha. You're referring to the second attempt, with a single VBO for positions, colours, and the matrix. I believe the buffer _vbo is still bound. Anyway, I tried rebinding it, still no drawing (see the commented out lines above... I tried with and without calling _program->enableAttributeArray(_matrixAttr) and _func330->glVertexAttribDivisor(_matrixAttr, 1) again) – Upbuild

I think you don't put things in the right order : in paintGL 1)vbo.bind vbo.write vbo.relase 2) program.bind vao.bind <draw your stuff> vao.release program.release. Remember OpenGL is a state machine: allways leave the state as you found it, if not you will cry! Verify your code with a simple glDrawArray to go step by step! Personaly I write dynamicaly (DynamicDraw) in vbo only when something change (not at every rendering!), I use multiple vbo and one vao and everything is fine. – Muley