Layered COLLADA Loader for OpenGL3.1+ Context

Question

What layers would aid in implementing this efficiently? Preferably such that I am able to get these three components listed below up with the early layers and code the rest as I go.

I am writing a COLLADA loading library to use with my graphics engine. So of course as soon as I began I notice this seems to be a rather large task. I now intend to write the library to use different layers.

I do not yet understand COLLADA. I'm assuming there may be:

• Geometry layer
• Shader layer
• Texure Layer

Even these I'm unsure of, may need to be part of a different sort of layering scheme.

Side note, I will most likely be using irrXML or this interesting XML library I just read about here to read the files unless advised otherwise.

Substratosphere

Answer 1

Are you aware of Open Asset Import Library (http://assimp.sourceforge.net/)? It has quite decent support for Collada loading and targets use in graphics engines. Writing a collada loader on your own is not only hard, it also takes ages to get it stable enough for productive use, mostly because the format is terribly complex (or, as the primary author of Assimp's Collada loader jokingly put is: almost turing-complete) and every exporter has its own strange habits.

If you need additional features, you can base your work on Assimp's Collada loader and add just what you need (BSD license, you may take it as long as you attribute the source).

Note: I am affliated with the project and thus not unbiased. But after the horrible experiences we made with writing our own Collada loader, I just feel like having to warn you …

Side note, I will most likely be using irrXML

We use it too. Should you really start from scratch, don't. Use a DOM parser, such as TinyXML. Collada is complex but very well-defined, a SAX parser just hardens the work since you have to build the document tree on your own (and you *will** need a data structure to resemble the full document - Collada elements are highly interconnected. To read something meaningful out of them, various cross references need to be resolved).

Cathryncathy

Can you suggest appropriate layers for interfacing OpenAsset loader with a rendering engine supporting the full range of features? I have heard of the library but am always concerned about overhead that I am not aware of and multithreading/streaming difficulties. If you have any words on that I would appreciate it. – Substratosphere

Regarding building the tree, this is mainly why I thought I would end up needing to write my own loader or at least translator from another loader which..seemed redundant. I am going to need data to be packed in specific ways from time to time as I load it, primarily for OpenCL. – Substratosphere

I don't get what you mean by 'layer'. Assimp reads a file at once, gathers everything it understands and converts it to a unified output data structure so the caller just need to process this (in-memory) format. – Cathryncathy

It will be easy to convert Assimp's output data to any representation you like it to have, be it OpenCl, OpenGl, whatever. If it lacks support for a certain Collada feature you need to support, you would need to modify the loader code. As I mentioned, you can simply base your work on it and attribute the source, that will be legally fine. Regarding overhead: Reading a complex XML file like a Collada file is overhead. Assimp is overhead (both in terms of memory consumption, image size and processing speed). If that overhead is too large for your application, consider [snip] – Cathryncathy

to chop the import part off from the main application: load your collada files once (regardless how you do that), convert them to your desired representation, save it to disk and have your app load this optimized blob. – Cathryncathy

Re-reading your question, I think that you should probably take a look at Collada first - how it looks, what it contains. To me, your 'layers' don't seem like a good model to describe a Collada file. Surely there are shaders, textures and geometry encoded somewhere in the file, but they are not totally separate (they are interconnected) and they are not specified in a way that I would ever call 'layer'. – Cathryncathy

Thanks much for your input, the layering is only due to the nature of my engine rather than Collada; this being why I needed advice from someone more familliar with the spec. Our layers are also very interconnected/dependant though I have no idea if they bear any remote similarity. I am only saying, with the way Collada works if you had to translate any render enhancing/altering data into a layered system what layers would be most appropriate? ....You have definitely pointed me in a good direction here. I will just get them into our model database in native format using Assimp. :) – Substratosphere

I could probably alter the loader code to use appropriate datastructures,... but then if I just translate them I wont have to worry about threading issue with --noboost – Substratosphere

"unified output data structure" refering to this concept, doesn't a variety exist within the structure representing many categories of information? This would be what the layers, which I should perhaps rename channels, aim to abstract. – Substratosphere

Well, what you get is meshes (raw vertices with a set of optional data streams), mesh adjacency (scenegraph, transformation hierarchy), animation channels and materials (which are just a key-value store representing roughly what was read from the original file). These different channels are more or less independent, although they, of course, reference each other. – Cathryncathy

And don't worry about noboost. If you use the C++ interface and create and use one Importer instance per thread, it will be all fine. No threading issues (I'm unsure if this is explicitly stated in the documentation, but rest assured that it wont change in future versions). – Cathryncathy

+1 for: "Collada can do everything and the kitchen sink. In fact I suspect it is a Turing complete file format." :D – Rendarender