Greenish image with BGRA to YUV444 conversion using DirectX11 pixel shader

Question

  I'm new to HLSL. I am trying to convert color space of an image captured using DXGI Desktop Duplication API from BGRA to YUV444 using texture as render target.
  I have set my pixel shader to perform the required transformation. And taking the 4:2:0 sub-sampled YUV from render target texture and encoding it as H264 using ffmpeg, I can see the image.
  The problem is - it is greenish.
  The input color information for the shader is of float data type but the coefficient matrix available for RGB to YUV conversion assumes integer color information.
  If I use clamp function and take the integers out of input color, I'm losing the accuracy.
  Any suggestions and directions are welcome. Please let me know if any other information helps.
  I suspect the Pixel shader I wrote, As I am working with it for the first time. Here is the pixel shader.

float3 rgb_to_yuv(float3 RGB)
{
    float y = dot(RGB, float3(0.29900f, -0.16874f, 0.50000f));
    float u = dot(RGB, float3(0.58700f, -0.33126f, -0.41869f));
    float v = dot(RGB, float3(0.11400f, 0.50000f, -0.08131f));
    return float3(y, u, v);
}
float4 PS(PS_INPUT input) : SV_Target
{
    float4 rgba, yuva;
    rgba = tx.Sample(samLinear, input.Tex);
    float3 ctr = float3(0, 0, .5f);
    return float4(rgb_to_yuv(rgba.rgb) + ctr, rgba.a);
}

  The render target is mapped to CPU readable texture and copying the YUV444 data into 3 BYTE arrays and supplying to ffmpeg libx264 encoder. The encoder writes the encoded packets to a video file.
  Here I'm taking for each 2X2 matrix of pixels one U(Cb) and one V(Cr) and 4 Y values.
  I retrieve the yuv420 data from texture as :

for (size_t h = 0, uvH = 0; h < desc.Height; ++h)
{
    for (size_t w = 0, uvW = 0; w < desc.Width; ++w)
    {
        dist = resource1.RowPitch *h + w * 4;
        distance = resource.RowPitch *h + w * 4;
        distance2 = inframe->linesize[0] * h + w;
        data = sptr[distance + 2 ];
        pY[distance2] = data;
        if (w % 2 == 0 && h % 2 == 0)
        {
            data1 = sptr[distance + 1];
            distance2 = inframe->linesize[1] * uvH + uvW++;
            pU[distance2] = data1;
            data1 = sptr[distance ];
            pV[distance2] = data1;
        }
    }
    if (h % 2)
        uvH++;
} 

EDIT1: Adding the Blend state desc :

D3D11_BLEND_DESC BlendStateDesc;
    BlendStateDesc.AlphaToCoverageEnable = FALSE;
    BlendStateDesc.IndependentBlendEnable = FALSE;
    BlendStateDesc.RenderTarget[0].BlendEnable = TRUE;
    BlendStateDesc.RenderTarget[0].SrcBlend = D3D11_BLEND_SRC_ALPHA;
    BlendStateDesc.RenderTarget[0].DestBlend = D3D11_BLEND_INV_SRC_ALPHA;
    BlendStateDesc.RenderTarget[0].BlendOp = D3D11_BLEND_OP_ADD;
    BlendStateDesc.RenderTarget[0].SrcBlendAlpha = D3D11_BLEND_ONE;
    BlendStateDesc.RenderTarget[0].DestBlendAlpha = D3D11_BLEND_ZERO;
    BlendStateDesc.RenderTarget[0].BlendOpAlpha = D3D11_BLEND_OP_ADD;
    BlendStateDesc.RenderTarget[0].RenderTargetWriteMask = D3D11_COLOR_WRITE_ENABLE_ALL;
    hr = m_Device->CreateBlendState(&BlendStateDesc, &m_BlendState);
FLOAT blendFactor[4] = {0.f, 0.f, 0.f, 0.f};
    m_DeviceContext->OMSetBlendState(nullptr, blendFactor, 0xffffffff);
    m_DeviceContext->OMSetRenderTargets(1, &m_RTV, nullptr);
    m_DeviceContext->VSSetShader(m_VertexShader, nullptr, 0);
    m_DeviceContext->PSSetShader(m_PixelShader, nullptr, 0);
    m_DeviceContext->PSSetShaderResources(0, 1, &ShaderResource);
    m_DeviceContext->PSSetSamplers(0, 1, &m_SamplerLinear);
    m_DeviceContext->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);

EDIT2 : The value of Y U V when calculated on CPU:45 200 170 and values after pixel shader which involves floating point calculations:86 141 104. The corresponding R G B:48 45 45. What could be making the difference?

Wolfram

am also facing same issue – Anomie

Why is there an offset of 0.5 for V, but not U? That seems asymmetric – Fertility

@harold , Tried with (0, 0.5, 0.5) but problem remains same. The greenish effect changed to some other color effect. – Wolfram

Where did you get your rgb_to_yuv function? The Wikipedia article en.wikipedia.org/wiki/… shows completely other coefficients. – Methenamine

I picked the coefficients from codegists.com/code/yuv-rgb888 . Also tried the wiki coefficients. Problem still remains. – Wolfram

@harold , Please have a look at the additional information I mentioned – Wolfram

@Methenamine , Please have a look at the additional information I mentioned – Wolfram

@PRSingh , Please have a look at the additional information I mentioned – Wolfram

@BalakrishnaAvulapati I am having trouble with this myself. Is there any chance you could post the basic source of what you made? – Suspicious

@Suspicious please find the working code and shaders at github.com/bavulapati/… . Hope this helps – Wolfram

@BalakrishnaAvulapati, I have looked at your sample. It's really useful. I have a question though, now I'm left with two channels, chroma, and luma respectively, is there a way to pack these into single ID3D11Texture2D texture so that I can easily feed the same to MediaFoundation MFT? – Advocaat

@RameshKambadaasan Use NV12 texture and fill it with the data – Wolfram

Answer 1

It looks like your matrix is transposed.

According to: www.martinreddy.net/gfx/faqs/colorconv.faq under [6.4] ITU.BT-601 Y'CbCr:

Y'= 0.299*R' + 0.587*G' + 0.114*B'
Cb=-0.169*R' - 0.331*G' + 0.500*B'
Cr= 0.500*R' - 0.419*G' - 0.081*B'

You misinterpreted the behavior of numpy.dot in the source you copied.

Also, it looks like @harold is correct, you should be offsetting both U and V.

Tauro

Answer 2

Based on this Wikipedia article, to convert RGB -> YUV444 (BT.601) you should use this function:

float3 RGBtoYUV(float3 c)
{
       float3 yuv;
       yuv.x = dot(c, float3(0.299, 0.587, 0.114));
       yuv.y = dot(c, float3(-0.14713, -0.28886, 0.436));
       yuv.z = dot(c, float3(0.615, -0.51499, -0.10001));
       return yuv;
}

Also, what's the format of the texture that you load into your shader? Considering that you are using float4 rgba, yuva;, did you convert BGRA -> RGBA first?

Herlindaherm

I am accessing the (r,g,b) tuple using the hlsl color semantics (rgba.r, rgba.g, rgba.b). BGRA -> RGBA conversion is not required – Wolfram

If your texture format is BGRA, rgba.r doesn't return R channel, it returns B channel. You can read about this here. So in order to get RGBA from you BGRA texture, you have to do the following conversion: float4 color = float4(rgba.b, rgba.g, rgba.r, rgba.a);. – Herlindaherm