error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #1 of opcode #86 (counts are 1-based)

Question

I'm absolutely stumped as well as my instructors/lab-assistants.

For some reason, the following HLSL code is returning this in the output window:

error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #1 of opcode #86 (counts are 1-based).

Here's the function in the HLSL causing the issue:

// Projects a sphere diameter large in screen space to calculate desired tesselation factor
float SphereToScreenSpaceTessellation(float3 p0, float3 p1, float diameter)
{
float3 centerPoint = (p0 + p1) * 0.5f;

float4 point0 = mul( float4(centerPoint,1.0f) , gTileWorldView);

float4 point1 = point0;
point1.x += diameter;

float4 point0ClipSpace = mul(point0, gTileProj);
float4 point1ClipSpace = mul(point1, gTileProj);

point0ClipSpace /= point0ClipSpace.w;
point1ClipSpace /= point1ClipSpace.w;

point0ClipSpace.xy *= gScreenSize;
point1ClipSpace.xy *= gScreenSize;

float projSizeOfEdge = distance(point0ClipSpace, point1ClipSpace);

float result = projSizeOfEdge / gTessellatedTriWidth;

return clamp(result, 0, 64);
}

I've narrowed it down to the point where it may be the "mul" intrinsic. We've taken everything out of the code and tried to return out a temporary variable like this, and it works fine:

float SphereToScreenSpaceTessellation(float3 p0, float3 p1, float diameter)
{

float temp = 0;

float3 centerPoint = (p0 + p1) * 0.5f;

float4 point0 = mul( float4(centerPoint,1.0f) , gTileWorldView);

float4 point1 = point0;
point1.x += diameter;

float4 point0ClipSpace = mul(point0, gTileProj);
float4 point1ClipSpace = mul(point1, gTileProj);

point0ClipSpace /= point0ClipSpace.w;
point1ClipSpace /= point1ClipSpace.w;

point0ClipSpace.xy *= gScreenSize;
point1ClipSpace.xy *= gScreenSize;

float projSizeOfEdge = distance(point0ClipSpace, point1ClipSpace);

float result = projSizeOfEdge / gTessellatedTriWidth;

return temp;
//return clamp(result, 0, 64);
}

If anyone is wondering:

gTileWorldView, gTileProj are float4x4's in a .hlsli file
gScreenSize is a float2 in a .hlsli file.
gTessellatedTriWidth is a float in a .hlsli file.

The following function is as states in a 2011 NVidia shader at : http://dx11-xpr.googlecode.com/svn/trunk/XPR/Media/Effects/TerrainTessellation.fx

I tried to copy and paste their solution replacing their variables with the one above, and the same error listed happens.

I'm absolutely stumped and I need assistance in order to do this assignment, please help.

Storehouse

I don't know how the toolchain works here, but if the HLSL is compiled by an external compiler then read into DirectX is there any chance that that separate HLSL compiler is too old, or too new? If this is being compiled by a Microsoft compiler, would it be worth kicking off a support incident to find out if it's a bug in their compiler - particularly if you can demonstrate it won't compile NVidia's example code? – Prelect

I'm running out of options, so I might have to kick this off to support. I'm using the latest HLSL and Shader Model of 5.0, so I don't think having things outdated would be the problem. Thanks for the help though ^^ – Storehouse

The problem will probably be in the shader that calls this. I'm guessing you're making a hull shader? I found that 'out' parameters in hull shaders often lead to internal compiler errors, but returning structs with semantics is fine. – Hobard

Answer 1

Check out this line:

point0ClipSpace.xy *= gScreenSize;

Is gScreenSize a float2? I do not believe you can scalar multiply a vec by any vec type.

Heliograph