Use SCNTechnique with Metal Shaders in Swift Playgrounds

Question

I want to use a SCNTechnique with Metal shaders in Swift Playgrounds on a SCNView. Therefore I tried compiling my shaders with the following code:

guard let metalDevice = sceneView.device else {
    return
}

if let path = Bundle.main.path(forResource: "distortTechnique", ofType: "metal") {
    do {
        let contents = try String(contentsOf: URL(fileURLWithPath: path))
        do {
            try metalDevice.makeLibrary(source: contents, options: nil)
        } catch { error
            print(error)
        }
    } catch {
        // contents could not be loaded
    }
}

The file is loaded but I get the following compiler error:

Error Domain=MTLLibraryErrorDomain Code=3 "Compilation failed: 

program_source:11:10: fatal error: 'SceneKit/scn_metal' file not found
#include <SceneKit/scn_metal>
         ^
" UserInfo={NSLocalizedDescription=Compilation failed: 

program_source:11:10: fatal error: 'SceneKit/scn_metal' file not found
#include <SceneKit/scn_metal>
         ^
}

So it seems like the »scn_metal« library can't be found. Any idea how I can solve this? The same setup is running perfectly fine in an iOS App project.

Here is the shader code:

#include <metal_stdlib>
using namespace metal;
#include <SceneKit/scn_metal>

struct custom_vertex_t
{
    float4 position [[attribute(SCNVertexSemanticPosition)]];
};

constexpr sampler s = sampler(coord::normalized,
                              address::repeat,
                              filter::linear);

struct out_vertex_t
{
    float4 position [[position]];
    float2 uv;
    float time;
    float random;
};

vertex out_vertex_t pass_through_vertex(custom_vertex_t in [[stage_in]],
                                        constant SCNSceneBuffer& scn_frame [[buffer(0)]])
{
    out_vertex_t out;
    out.position = in.position;
    out.uv = float2((in.position.x + 1.0) * 0.5 , (in.position.y + 1.0) * -0.5);
    out.time = scn_frame.time;

    return out;
};

fragment half4 pass_through_fragment(out_vertex_t vert [[stage_in]],
                            texture2d<float, access::sample> colorSampler [[texture(0)]])
{

    float4 fragment_color = colorSampler.sample( s, vert.uv);
    return half4(fragment_color);
};

fragment half4 distort_fragment(out_vertex_t vert [[stage_in]],
                                              texture2d<float, access::sample> colorSampler [[texture(0)]])
{
    float multiplier = sin(vert.time * 0.5);
    float effectRadius = 0.75;
    float effectAmount = 360. * 2.;
    float effectAngle = multiplier * effectAmount;
    effectAngle = effectAngle * M_PI_F / 180;

    float2 resolution = float2(colorSampler.get_width(), colorSampler.get_height());
    float2 center = float2(0.5, 0.5);//iMouse.xy / iResolution.xy;

    float2 uv = vert.position.xy / resolution - center;
    float len = length(uv * float2(resolution.x / resolution.y, 1.));
    float angle = atan2(uv.y, uv.x) + effectAngle * smoothstep(effectRadius, 0., len);
    float radius = length(uv);

    float4 fragment_color = colorSampler.sample(s, float2(radius * cos(angle), radius * sin(angle)) + center);

    return half4(fragment_color);

};

Attached is an image of the playground.

Thanks!

Housewares

Answer 1

I was able to get a metal shader working as an SCNProgram in a playground by including the contents of SCNMetalDefines at the top of the metal file. I'm not sure whether scn_metal has more to it than these defines. This had everything I needed for a vertex/fragment pipeline though. Could you post the code of your shaders? Or at least, could you tell us what the shader references in scn_metal?

#include <metal_stdlib>
using namespace metal;

#ifndef __SCNMetalDefines__
#define __SCNMetalDefines__

enum {
SCNVertexSemanticPosition,
SCNVertexSemanticNormal,
SCNVertexSemanticTangent,
SCNVertexSemanticColor,
SCNVertexSemanticBoneIndices,
SCNVertexSemanticBoneWeights,
SCNVertexSemanticTexcoord0,
SCNVertexSemanticTexcoord1,
SCNVertexSemanticTexcoord2,
SCNVertexSemanticTexcoord3
};

// This structure hold all the informations that are constant through a render pass
// In a shader modifier, it is given both in vertex and fragment stage through an argument named "scn_frame".
struct SCNSceneBuffer {
float4x4    viewTransform;
float4x4    inverseViewTransform; // transform from view space to world space
float4x4    projectionTransform;
float4x4    viewProjectionTransform;
float4x4    viewToCubeTransform; // transform from view space to cube texture space (canonical Y Up space)
float4      ambientLightingColor;
float4        fogColor;
float3        fogParameters; // x:-1/(end-start) y:1-start*x z:exp
float2      inverseResolution;
float       time;
float       sinTime;
float       cosTime;
float       random01;
};

// In custom shaders or in shader modifiers, you also have access to node relative information.
// This is done using an argument named "scn_node", which must be a struct with only the necessary fields
// among the following list:
//
// float4x4 modelTransform;
// float4x4 inverseModelTransform;
// float4x4 modelViewTransform;
// float4x4 inverseModelViewTransform;
// float4x4 normalTransform; // This is the inverseTransposeModelViewTransform, need for normal transformation
// float4x4 modelViewProjectionTransform;
// float4x4 inverseModelViewProjectionTransform;
// float2x3 boundingBox;
// float2x3 worldBoundingBox;

#endif /* defined(__SCNMetalDefines__) */

Douglass

Thank you Oliver! I found this file: github.com/theos/sdks/blob/master/iPhoneOS11.2.sdk/System/… and added the SCNMetalDefines. It is compiling now, but still not working. The scene now just stays pink. – Housewares

Could you post the rest of the playground? I think one of the trickier things about playgrounds is the limited debugging compared to a full project. You don't have the option of using GPU frame capture and so on. – Douglass

Sure! Here's the link: dropbox.com/s/gpe7jzs882scl3b/… – Housewares

I have to add that the final goal is to use the whole thing on an iPad, but I created the mac playground for easier debugging. – Housewares

Answer 2

I finally got it working by precompiling the metal library of an actual app project and then use the result default.metallib with the included shader functions in the playground.

This seems necessary as the SCNTechnique documentation for the vertex and frag,ent »These functions must exist in the app’s default Metal library.«

A bit hacky but works for my purpose.

Concrete steps: - create a new app project for the selected platform - setup SCNTechnique plist with all definitions - add .metal file with shader code - archieve the project - open the archieved bundle and copy the default.metallib file into your playground Resources folder - now just set your technique on the SCNView – profit.

Thanks again to Oliver for your help!

Housewares

Glad you got it working. It's a shame that you can't edit the metal shaders in the playground though. As you say, it must be an SCNTechnique requirement. SCNProgram runs fine compiling the shaders from a string. One minor suggestion on your code: you don't need to nest do/catch blocks inside one-another. You can have multiple try calls in a single do/ catch block. The first one to fail will break the do block and invoke the catch handler with the appropriate error. – Douglass

Yup, that's definitely a shame. Maybe there's some way to inject code in the default library at runtime. Will try it again when I got some more time on my hands. And thanks for the heads up with the do/catch blocks. I noticed that as well while fiddling around. – Housewares