Vulkan texture rendering on multiple meshes

Question

I am in the middle of rendering different textures on multiple meshes of a model, but I do not have much clues about the procedures. Someone suggested for each mesh, create its own descriptor sets and call vkCmdBindDescriptorSets() and vkCmdDrawIndexed() for rendering like this:

    // Pipeline with descriptor set layout that matches the shared descriptor sets
vkCmdBindPipeline(...pipelines.mesh...);
...
// Mesh A
vkCmdBindDescriptorSets(...&meshA.descriptorSet... );
vkCmdDrawIndexed(...);
// Mesh B
vkCmdBindDescriptorSets(...&meshB.descriptorSet... );
vkCmdDrawIndexed(...);

However, the above approach is quite different from the chopper sample and vulkan's samples that makes me have no idea where to start the change. I really appreciate any help to guide me to a correct direction.

Cheers

Transportation

And what method do those samples use? – F

Answer 1

You have a conceptual object which is made of multiple meshes which have different texturing needs. The general ways to deal with this are:

1. Change descriptor sets between parts of the object. Painful, but it works on all Vulkan-capable hardware.

2. Employ array textures. Each individual mesh fetches its data from a particular layer in the array texture. Of course, this restricts you to having each sub-mesh use textures of the same size. But it works on all Vulkan-capable hardware (up to 128 array elements, minimum). The array layer for a particular mesh can be provided as a push-constant, or a base instance if that's available.

   Note that if you manage to be able to do it by base instance, then you can render the entire object with a multi-draw indirect command. Though it's not clear that a short multi-draw indirect would be faster than just baking a short sequence of drawing commands into a command buffer.

3. Employ sampler arrays, as Sascha Willems suggests. Presumably, the array index for the sub-mesh is provided as a push-constant or a multi-draw's draw index. The problem is that, regardless of how that array index is provided, it will have to be a dynamically uniform expression. And Vulkan implementations are not required to allow you to index a sampler array with a dynamically uniform expression. The base requirement is just a constant expression.

   This limits you to hardware that supports the shaderSampledImageArrayDynamicIndexing feature. So you have to ask for that, and if it's not available, then you've got to work around that with #1 or #2. Or just don't run on that hardware. But the last one means that you can't run on any mobile hardware, since most of them don't support this feature as of yet.

   Note that I am not saying you shouldn't use this method. I just want you to be aware that there are costs. There's a lot of hardware out there that can't do this. So you need to plan for that.

F

thank you so much for the advice and that would be very helpful to me. – Transportation

Regarding 1.: Does that mean creating multiple VkDescriptorSets that contain different VkImageViews and switch between them using vkCmdBindDescriptorSets while filling a command buffer, before issuing the respective vkCmdDraw* draw commands for the individual meshes? – Jubbah

To answer my own question - that approach (1) appears to work fine, for hardware that doesn't support shaderSampledImageArrayDynamicIndexing (in my case, a Vivante GC7000Lite that is part of the NXP i.MX8M processor). Only a single VkSampler is required, and care must be taken to use appropriately-sized descriptor set pools. The individual descriptor sets can also contain individual uniform buffer objects which could contain different model matrices for the individual meshes/objects. – Jubbah

Answer 2

The person that suggested the above code fragment was me I guess ;)

This is only one way of doing it. You don't necessarily have to create one descriptor set per mesh or per texture. If your mesh e.g. uses 4 different textures, you could bind all of them at once to different binding points and select them in the shader.

And if you a take a look at NVIDIA's chopper sample, they do it pretty much the same way only with some more abstraction.

The example also sets up descriptor sets for the textures used :

VkDescriptorSet *textureDescriptors = m_renderer->getTextureDescriptorSets();

binds them a few lines later :

VkDescriptorSet sets[3] = { sceneDescriptor, textureDescriptors[0], m_transform_descriptor_set };
vkCmdBindDescriptorSets(m_draw_command[inCommandIndex], VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, 3, sets, 0, NULL);

and then renders the mesh with the bound descriptor sets :

vkCmdDrawIndexedIndirect(m_draw_command[inCommandIndex], sceneIndirectBuffer, 0, inCount, sizeof(VkDrawIndexedIndirectCommand));
vkCmdDraw(m_draw_command[inCommandIndex], 1, 1, 0, 0);

If you take a look at initDescriptorSets you can see that they also create separate descriptor sets for the cubemap, the terrain, etc.

The LunarG examples should work similar, though if I'm not mistaken they never use more than one texture?

Cholinesterase

Thank you so much for your reply. The m_texture_descriptor_sets is supposed to be the descriptor sets that handles the materials/textures of chopper model, but its size looks like being available only for one submesh as m_texture_descriptor_sets = (VkDescriptorSet*)malloc(sizeof(VkDescriptorSet));. I am wondering if I understand that properly. – Transportation

I haven't dug too deep into the chopper sample, but if you take a look at the fragment shader you can see that it uses a sampler array : layout(set = 1, binding = 0) uniform sampler2D tex[16]; So I guess submeshes just use a different index into that sampler array to select between the textures. That's another way of achieving different textures per (sub)mesh. – Cholinesterase

Thanks for the hint. I am now working on LunarG example's Mesh project about the multiple descriptor sets. I created a VkDescriptorSet array to bind each mesh's textures and tried to allocate memory for them as VkDescriptorSet* texDescriptorSets = (VkDescriptorSet*)malloc(sizeof(VkDescriptorSet) * g_MeshCount);, however, it seems the device object has a strict limitation for its size and therefore I have a device-out-of-memory assertion when calling vkAllocateDescriptorSets() function. It would be very kind of you to tell me if I am doing that right. – Transportation

@MichaelWei: So if you want a way that will work across all hardware, you must use different descriptor sets for different objects. – F

@MichaelWei : If you get an out of memory result on allocating descriptor sets make sure you have set the correct pool sizes (see here). If you want to use 4 combined image samplers for your textures you also have to request the at least 4 * (no. of descriptor sets that use this layout ) VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLERs from the descriptor pool. – Cholinesterase

For your example (there are 4 combined images samplers for textures), should the request for descriptor sets to be descriptorPoolSize(VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 4) or repeating descriptorPoolSize(VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 1) for four times? – Transportation

@MichaelWei: First, that's a new question (presumably, "How do I allocate a descriptor pool for sampler arrays?" or something to that effect). Second, what do you mean by descriptorPoolSize? That's not a valid Vulkan identifier. So when you ask your new question, you should show actual Vulkan code. – F

@NicolBolas: The VkDescriptorPoolSize is a valid identifier in vulkan.h – Transportation

@MichaelWei: Yes, but it's not a function that takes two parameters. Also, it's not spelled "descriptorPoolSize". My point is that you should use actual Vulkan syntax, not abbreviations that we nave to decipher. That makes it difficult to search through the Vulkan specification for answers. – F

@MichaelWei: In case you're still interested, I found some time to add a new example to my repository that renders a complete scene with multiple meshes using different textures and materials. It also demonstrates use of multiple bound descriptor sets and push constants for passing material properties. You can find it here – Cholinesterase

@SaschaWillems: Hi Sascha, I have figured out multiple texture rendering few weeks ago. Thanks for your reply and help though. – Transportation