Passing values between SCNShadable entry points

Question

In an OpenGL program you typical declare something like this in a vertex shader

varying bool aBooleanVariable;

and then read the value in the fragment shader. How do you do this within the framework of an SCNShadable entry point? For example from SCNShaderModifierEntryPointGeometry to SCNShaderModifierEntryPointFragment.

Receiving the argument seems to be defined by using the pragma arguments, I provide my test SCNShaderModifierEntryPointFragment to illustrate.

#pragma arguments
bool clipFragment

#pragma body

if (clipFragment) {
    discard_fragment();
}

However, the arguments pragma doesn't work for outputting the value in the SCNShaderModifierEntryPointGeometry entry point.

I found an article that suggests that it can be done with GLSL syntax, but I was trying to find the Metal way and I wasn't even able to reproduce the result.

Comminate

Answer 1

Forced to use GLSL

Turns out you can get it to work if you switch the SCNView Rendering API to OpenGL ES. Then it will accept GLSL code including the varying qualifier.

Geometry entry point

varying float clipFragment;

#pragma body
clipFragment = 1.0;

Fragment entry point

varying float clipFragment;

#pragma body
if (clipFragment > 0.0) {
    discard;
}

Update from Apple

I opened a ticket with Apple and got the following response -

That’s correct. The SceneKit team has opened up a formal enhancement request (bug #28206462) to investigate adding this capability to Metal. Until then, you’ll have to opt into using the OpenGL renderer.

Comminate

PBR? Can you define terms? – Comminate

Physically Based Rendering / Shading. It's a lighting model a bit more elaborate than Blinn/phong etc. For some reason it works only with metal, no idea why it wouldn't be compatible with OpenGL/GLSL. I encountered a really weird bug that i think could be solved by passing a custom varying with _geometry.texcoords[0], but i found out varyings only work with glsl :((( – Slam

Ah. Yes, wasn't familiar with the abbreviation, and yes it doesn't work. We can hope they add it – Comminate

Your question reads “how do I use something like varyings with Metal SceneKit shaderModifiers” and your answer comes down to “don't use Metal”, which IMHO is not a valid answer. I came to this question hoping to find workarounds like @lock's or — ideally — a minimal implementation of copying SceneKit's common profile shader into a custom SCNProgram and configuring it to work properly with geometry/textures/shadows/etc. So IMHO: Answering your own question is fine here on SO, but having that answer be “you can't do it” is not okay. Downvoting for this reason. – Acanthopterygian

@SlippD.Thompson I'm sorry you feel that way. You can do it with OpenGL which is the stated position of Apple Developer Support until they extend Metal. Offering SCNProgram as a solution is substantially more trouble than SCNShadable and loses much of the advantage of SCNShadable. Personally, this answer is useful in part because it saves people from wasting time trying to make it work. But thanks for the advice and the down vote. – Comminate

Hey, I'm honestly sorry for the downvote, but it just isn't an applicable answer to me. For me, using varyings is easy & obvious— it's OpenGL 101. But doing the same in SceneKit Metal is the challenge, how I read the question, and why I came here. Also, using OpenGL isn't an option for my project— PBR, better performance, and the market advantage Apple liking to feature apps that use Metal are all necessities in my case. – Acanthopterygian

SCNProgam is likely in your future. I considered it as well, but I really didn't want to give up the free stuff and write a lot of boiler-plate so I could make a 2-line change to one fragment shader. I would really prefer it if they extended Metal and made this Question/Answer obsolete. – Comminate

@SlippD.Thompson good adds on the tags for the article though. Maybe someone will provide a better answer. – Comminate

Answer 2

I had a similar problem and managed to figure out the solution. I'm not sure how far back this works, but it works on XCode 9 with iOS 11. Here's an example of looking up a custom cubemap texture using a varying float3 for the coordinate.

The geometry modifier:

#include <metal_stdlib>

#pragma varyings
float3 cubemapCoord;

#pragma body
out.cubemapCoord = _geometry.normal;

The surface modifier:

#include <metal_stdlib>

#pragma arguments
texturecube<float> texture;

#pragma body
constexpr sampler s(filter::linear, mip_filter::linear);
_surface.diffuse = texture.sample(s, in.cubemapCoord);

And the Swift code for assigning the material:

do {
    let loader = MTKTextureLoader.init(device: MTLCreateSystemDefaultDevice()!)
    let url = Bundle.main.url(forResource: "cubemap", withExtension: "pvr", subdirectory: "art.scnassets")!
    let texture = try loader.newTexture(URL: url, options: nil)
    let materialProperty = SCNMaterialProperty.init(contents: texture)
    geometry.setValue(materialProperty, forKey: "texture")
} catch {
    print(error.localizedDescription)
}

Johnjohna

This is the right answer at least for the current ios and xcode. Thanks for posting this, this issue was keeping me from switching to metal. The key parts are using the #include line in the geometry entry point, the #pragma varyings line and prefixing the variable with in./.out. – Stewpan

How did you discover to use in and out? Haven't seen that before (not even in Apple's Metal Shading Language Specification document). – Dewees

@Dewees I remember it took a lot of digging around to figure out how to make this work, but it was quite a while ago so I don't remember how I cobbled the pieces together. I sort of recall finding what I needed buried in some source code example from Apple, but I'm not sure. – Johnjohna

This morning I uncovered this gem. I'm still processing everything in there (including the pointer to access this file from Xcode) but once I've wrapped my head around it I'll probably add an additional answer here with up-to-date information. – Dewees

Answer 3

At some point you will bump into the limits of what can be done with SCNShadable and need to move onto SCNProgram. Doing so unfortunately means you need to provide a complete implementation of the vertex and fragment shaders. On the positive side, it is rather simple to pass a value from the vertex to the fragment shader. You simply add a variable to the struct you identify with the stage_in qualifier (assuming you're using Metal). But that's not really what you asked, and there is a workaround for that.

#pragma arguements as you noted, wont work in this case; it's for passing in constants that do not vary over a render pass.

Instead I'd suggest you look at repurposing one of the existing variables SceneKit uses in its shader. By now you've no doubt noticed SceneKit dumps its shader source code to stdout when it fails to compile. The Metal shader defines a commonprofile_io struct which is passed from the vertex to fragment shader, it includes: position, colour, texture coordinates, normals, etc. If you're not using vertex colours, then it's possible to use this to store 4 float values (rgba). Important to note that SceneKit optimises the shader for the geometry source being rendered; if your geometry source contains no vertex colours, then the commonprofile_io struct will not have a vertexColor variable.

Would be very interested to know of a better solution.

Chump

It is certainly difficult to find details as well. So far the best resources are the header files, the reference, and the dump of the shader source. This guy insinuates that he did it using GLSL syntax. ericberna.com/2015/03/20/SCNShadable.html – Comminate

At some point I'll bump into the limits... you mean immediately. :) – Comminate

Answer 4

I've struggling with this question too, and I've made a recent discovery that blew this thing wide open for me. It has aided me tremendously in grasping how SceneKit, shaders, shader modifiers and OpenGL/Metal work together and what tools/methods I have available when writing a shader modifier. Furthermore, I believe the currently accepted answer is no longer correct (fortunately! 🎉).

SceneKit provides default vertex and fragment Metal shaders. It is into these shaders that the shader modifiers are injected. You can find these default Metal shaders when you capture a GPU frame in Xcode. Double click on either commonprofile_vert or commonprofile_frag (both shaders are inside the same "file", as demonstrated by the same values in the Details column).

There is a lot of information in this file, and it demonstrates some of the inner workings of how SceneKit talks to the GPU when rendering. This is an example of the file (although this version is a couple of years old already it is still exemplary).

Notice how the following lines are injection places for the shader modifiers.

// DoLightModifier START

// DoLightModifier END

...

// DoGeometryModifier START

// DoGeometryModifier END

...

// DoSurfaceModifier START

// DoSurfaceModifier END

...

// DoFragmentModifier START

// DoFragmentModifier END

You will see that your shader modifiers have been placed here.

Look for #ifdef USE_EXTRA_VARYINGS. Any varying variables you declare in the geometry modifier show up as property on commonprofile_io structs. Of which you use the following instance: commonprofile_io out;.

This file also demonstrates why you need varyings for communication between the geometry and fragment shader. And why _surface is accessible in the fragment shader: the fragment modifier and surface modifier have the same scope.

I am absolutely stumped how none of this stuff is documented or even hinted at by Apple, save for some tucked away WWDC-slides probably. I have no idea how people without a couple of weeks of spare time are supposed to figure out how it all works.

Dewees