How can I get and set pixel values of an EmguCV Mat image?

Question

I'm using the EmguCV 3.0.0 wrapper to the OpenCV 3.0 library. I'm using the Mat class in a few places. Here's an example of a single channel, 8x8 image made of double values:

Mat image = new Mat(8, 8, DepthType.Cv64F, 1);

The Image<> class provides reasonable means for getting and setting pixel values, and the method is identical for the Matrix<> class, but it doesn't seem as obvious for the Mat class. The only way I've figured out how to set individual pixel is using a mask:

// set two pixel values, (0,0) to 9.0, (2, 3) to 42.0

Matrix<byte> mask = new Matrix<byte>(8,8);
mask.Data[0, 0] = 1;
image.SetTo(new MCvScalar(9.0), mask);

mask = new Matrix<byte>(8,8);
mask.Data[2, 3] = 1;
image.SetTo(new MCvScalar(42.0), mask);

This is feels like it should be two lines, not six, so I feel like I'm missing something. Things get even more complicated when the Mat is more than one channel, because Matrix<> is only 2D, so the mask must be used to set the pixel on each channel.

I cannot afford the time or memory to set pixels this way. How can I set pixels with a single method call?

Tarentarentum

image[0,0]=9; should do it – Prakash

That's what I expected as well. But it doesn't work. Here's the error: "Cannot apply indexing with [] to an expression of type `Emgu.CV.Mat'" – Tarentarentum

Answer 1

You can get elements from Mat by copying unmanaged memory blocks using DataPointer and converting managed to unmanaged types. Setting values is marshaling in the opposite direction.

For an example you can use such an extension class

public static class MatExtension
{
    public static dynamic GetValue(this Mat mat, int row, int col)
    {
        var value = CreateElement(mat.Depth);
        Marshal.Copy(mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, value, 0, 1);
        return value[0];
    }

    public static void SetValue(this Mat mat, int row, int col, dynamic value)
    {
        var target = CreateElement(mat.Depth, value);
        Marshal.Copy(target, 0, mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, 1);
    }
    private static dynamic CreateElement(DepthType depthType, dynamic value)
    {
        var element = CreateElement(depthType);
        element[0] = value;
        return element;
    }

    private static dynamic CreateElement(DepthType depthType)
    {
        if (depthType == DepthType.Cv8S)
        {
            return new sbyte[1];
        }
        if (depthType == DepthType.Cv8U)
        {
            return new byte[1];
        }
        if (depthType == DepthType.Cv16S)
        {
            return new short[1];
        }
        if (depthType == DepthType.Cv16U)
        {
            return new ushort[1];
        }
        if (depthType == DepthType.Cv32S)
        {
            return new int[1];
        }
        if (depthType == DepthType.Cv32F)
        {
            return new float[1];
        }
        if (depthType == DepthType.Cv64F)
        {
            return new double[1];
        }
        return new float[1];
    }
}

Then getting and setting value is possible by single method call

var row = 2;
var col = 1;
var mat = new Mat(3, 3, DepthType.Cv64F, 3);
mat.SetValue(row, col, 3.14);
var value = mat.GetValue(row, col);

Tests with 200000000 operations shows that dynamic type version can be up to ~2.5x slower than static.

public static double GetDoubleValue(this Mat mat, int row, int col)
{
    var value = new double[1];
    Marshal.Copy(mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, value, 0, 1);
    return value[0];
}

public static void SetDoubleValue(this Mat mat, int row, int col, double value)
{
    var target = new[] { value };
    Marshal.Copy(target, 0, mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, 1);
}

Unguiculate

Interesting use of dynamic, that looks very nice to use! My only problem is that dynamic can be ~10x to ~100x slower than statically typed alternatives. (see here and here.) Your extension methods should be on the fast end of that spectrum because the dynamic receiving type will rarely change. – Tarentarentum

Even so, optimized Get/SetValue() methods safely returning/receiving double should be possible, because there is a safe automatic conversion from any Emgu supported DepthType to double. – Tarentarentum

Thank you! I wasn't expecting that. If the EmguCV maintainer doesn't get around to supporting this better, I'll include an extension method like yours soon. – Tarentarentum

Bartosz, great answer. Just one thing instead of (row * mat.Cols + col) * mat ElementSize, I would use ((row * mat.Step) + (col * mat.ElementSize)) as the row step might not be equal to row * mat.Columns – Boughpot

At first, nice answer, helped me a lot. But I do not understand how this method deals with multiple channels. In your example you wrote var mat = new Mat(3, 3, DepthType.Cv64F, 3); shouldn't it return a double[] of size 3 containing a value for each channel? – Feckless

so how to change pixel at channel number 3 for example? – Yclept

@peterbence Just add 2*sz, where sz = 1 for Cv8U up to 8 for Cv64F (i.e. double) depending on the DepthType. – Danais

Also, @Boughpot is right. Bartosz's answer won't work on non-continuous matrices (i.e. submatrices) unless Aero's fix is used. – Danais

Answer 2

Based on the Bartosz Rachwal's great answer, I've tried to write it for OpenCvSharp:

    public static dynamic GetValue(this Mat mat, int row, int col)
    {
        var value = CreateElement(mat.Type());
        Marshal.Copy(mat.Data + (row * mat.Cols + col) * mat.ElemSize(), value, 0, 1);
        return value[0];
    }
    public static void SetValue(this Mat mat, int row, int col, dynamic value)
    {
        var target = CreateElement(mat.Type(), value);
        Marshal.Copy(target, 0, mat.Data + (row * mat.Cols + col) * mat.ElemSize(), 1);
    }
    private static dynamic CreateElement(MatType depthType, dynamic value)
    {
        var element = CreateElement(depthType);
        element[0] = value;
        return element;
    }
    private static dynamic CreateElement(MatType depthType)
    {
        switch (depthType)
        {
            case MatType.CV_8S:
                return new sbyte[1];
            case MatType.CV_8U:
                return new byte[1];
            case MatType.CV_16S:
                return new short[1];
            case MatType.CV_16U:
                return new ushort[1];
            case MatType.CV_32S:
                return new int[1];
            case MatType.CV_32F:
                return new float[1];
            case MatType.CV_64F:
                return new double[1];
            default:
                throw new NotImplementedException();
        }
    }

Southdown

Thank you for the effort, but I don't think this belongs here. OpenCvSharp is a completely different library than the one I'm asking about. If there isn't an existing question on OpenCvSharp that you can move this to, consider asking and answering your own question. – Tarentarentum

Both are OpenCV wrapper, not very different as you see. When I require a knowledge about something I don't only look for OpenCvSharp threads. EMGU, Cpp even Phton examples give me great deal of help. A new question would be waste of time I think. This is the same problem, and almost the same code. Anybody searching like me might make use of it. Also your discussions about dynamic is a good read, anybody that would use this should also read them. Thanks. – Southdown

With OpenCvSharp, you have the Mat::Set<T>(x,y,val) method to set the element at (x,y) to the value of val. You don't have such an accessor in Emgu wrapper. – Criminate

Answer 3

A better one.

• Tick "Allow unsafe code" in the project's Debug and Release configurations.
• Code:

public static class MatExtension
{
    public static void Set<T>(this Mat mat, int row, int col, T value) where T : struct 
       => unsafe {  _ = new Span<T>(mat.DataPointer.ToPointer(), mat.Rows * mat.Cols * mat.ElementSize)
            [row * mat.Cols + col] = value; }

    public static T Get<T>(this Mat mat, int row, int col) where T : struct 
       => unsafe { return new ReadOnlySpan<T>(mat.DataPointer.ToPointer(), mat.Rows * mat.Cols * mat.ElementSize)
            [row * mat.Cols + col]; }

    public static ReadOnlySpan<T> Get<T>(this Mat mat, int row, System.Range cols) where T : struct
    {
        unsafe
        {
           var span = new ReadOnlySpan<T>(mat.DataPointer.ToPointer(), mat.Rows * mat.Cols * mat.ElementSize);
           var (offset, length) = cols.GetOffsetAndLength(span.Length);
           return span.Slice(row * mat.Cols + offset, length);
        }
    }
}

Usage:

using var stats = new Mat();
using var centroids = new Mat();

var x = stats.Get<int>(i,(int)ConnectedComponentsTypes.Left);
var cxy = centroids.Get<double>(i, 0..1);
var cxy0 = cxy[0];

Fatness

How about the .Set<struct> variant? – Loveinamist

Some guy messed up with my answer (edited it and edit was accepted). So beware of this code guys, it's not my "original" one anymore. – Fatness

Answer 4

This solution https://mcmap.net/q/196672/-how-can-i-get-and-set-pixel-values-of-an-emgucv-mat-image for three color channels as requested by the user Quergo:

At first, nice answer, helped me a lot. But I do not understand how this method deals with multiple channels. In your example you wrote var mat = new Mat(3, 3, DepthType.Cv64F, 3); shouldn't it return a double[] of size 3 containing a value for each channel? – Quergo Nov 18 '19 at 22:30

public static class MatExtension
{
    public static dynamic GetValues(this Mat mat, int row, int col)
    {
        var value = CreateElement3Channels(mat.Depth);
        Marshal.Copy(mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, value, 0, 3);
        return value;
    }

    public static dynamic GetValue(this Mat mat, int channel, int row, int col)
    {
        var value = CreateElement3Channels(mat.Depth);
        Marshal.Copy(mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, value, 0, 3);
        return value[channel];
    }

    public static dynamic GetValue(this Mat mat, int row, int col)
    {
        var value = CreateElement(mat.Depth);
        Marshal.Copy(mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, value, 0, 1);
        return value[0];
    }

    public static void SetValues(this Mat mat, int row, int col, dynamic value)
    {
        Marshal.Copy(value, 0, mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, 3);
    }

    public static void SetValue(this Mat mat, int channel, int row, int col, dynamic value)
    {
        var element = GetValues(mat, row, col);
        var target = CreateElement(element, value, channel);
        Marshal.Copy(target, 0, mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, 3);
    }

    public static void SetValue(this Mat mat, int row, int col, dynamic value)
    {
        var target = CreateElement(mat.Depth, value);
        Marshal.Copy(target, 0, mat.DataPointer + (row * mat.Cols + col) * mat.ElementSize, 1);
    }

    private static dynamic CreateElement(dynamic element, dynamic value, int channel)
    {
        element[channel] = value;
        return element;
    }

    private static dynamic CreateElement(DepthType depthType, dynamic value)
    {
        var element = CreateElement(depthType);
        element[0] = value;
        return element;
    }

    private static dynamic CreateElement3Channels(DepthType depthType)
    {
        if (depthType == DepthType.Cv8S)
        {
            return new sbyte[3];
        }

        if (depthType == DepthType.Cv8U)
        {
            return new byte[3];
        }

        if (depthType == DepthType.Cv16S)
        {
            return new short[3];
        }

        if (depthType == DepthType.Cv16U)
        {
            return new ushort[3];
        }

        if (depthType == DepthType.Cv32S)
        {
            return new int[3];
        }

        if (depthType == DepthType.Cv32F)
        {
            return new float[3];
        }

        if (depthType == DepthType.Cv64F)
        {
            return new double[3];
        }

        return new float[3];
    }

    private static dynamic CreateElement(DepthType depthType)
    {
        if (depthType == DepthType.Cv8S)
        {
            return new sbyte[1];
        }

        if (depthType == DepthType.Cv8U)
        {
            return new byte[1];
        }

        if (depthType == DepthType.Cv16S)
        {
            return new short[1];
        }

        if (depthType == DepthType.Cv16U)
        {
            return new ushort[1];
        }

        if (depthType == DepthType.Cv32S)
        {
            return new int[1];
        }

        if (depthType == DepthType.Cv32F)
        {
            return new float[1];
        }

        if (depthType == DepthType.Cv64F)
        {
            return new double[1];
        }

        return new float[1];
    }
}

Naumachia

Did you just link another answer for the same question? We appreciate the expanded solution, but having 3 color channels was not important to the original question. – Anhydride

To quote another answer: "At first, nice answer, helped me a lot. But I do not understand how this method deals with multiple channels. In your example you wrote var mat = new Mat(3, 3, DepthType.Cv64F, 3); shouldn't it return a double[] of size 3 containing a value for each channel? – Quergo Nov 18 '19 at 22:30" So there is definitely interest in 3 color channels and the top answer has a three color image in it, which would be a mistake in this case. – Naumachia

Hi eng3ls, it was not clear from your answer that you were addressing the comment of another user. I recommend updating your answer explaining this. Note: users who vote on something cannot change their vote the next day unless the post is edited. – Anhydride

I edited my solution mentioning that I am addressing the comment of another user. Since I have not enough reputation to comment on the top solution can you write a comment there, that the question by Quergo has been answered... it's 2 years old, so I don't think it really matters, but you never know. However the user peter bence also asked the same question last year. – Naumachia

Answer 5

you can easily get and set the pixel value of the Mat image in two ways:

1- convert Mat image to Image<ColorType, DDepth>
2- use SetValue and GetValue directly from the Mat class.

1- convert Mat image to Image<> Format:

int row = 0;
int col = 1;
int channel = 6;
Mat Image = CvInvoke.Imread("path");
Image<Gray, Byte> ImageFormat = Image.ToImage<Gray, Byte>();
int pixVal = ImageFormat.Data[row, col, channel];

// for set value 

int numValue = 165;
ImageFormat.Data[row, col, channel] = numValue;

you can access to Mat Format of Image<> with:

ImageFormat.Mat;

2- you can set or get pixel values directly with SetValue and GetValue methods (these methods return an object, you have to convert the object to a number):

object pixVal = Image.Data.GetValue(row, col, channel);
float pixValue = Convert.ToInt32(pixVal);

// for set value:

float setPixVal = 159;
Image.Data.SetValue(setPixVal, row, col, channel);

Falchion