Confusion on YUV NV21 conversion to RGB

Question

According to http://developer.android.com/reference/android/graphics/ImageFormat.html#NV21, NV21 is the default used format.

There are quite a number of code on web regarding YUV NV21 to RGB conversion. However, when I go through the code, I doubt on the correctness of the code.

The first component V should come first, followed by first component U

According to http://wiki.videolan.org/YUV#NV21, NV21 is like NV12, but with U and V order reversed: it starts with V. However, when I went through the code implementation

• http://pastebin.com/T0my7zSc - It assumes U comes first
• https://mcmap.net/q/162268/-android-how-to-display-camera-preview-with-callback - It assumes U comes first too
• https://mcmap.net/q/298542/-converting-yuv-gt-rgb-image-processing-gt-yuv-during-onpreviewframe-in-android - It assmes U comes first too

R should be the most significant position According implementation of int argb in Color.java, R suppose to be at the most significant position. However, I went through the following code implementation

• http://pastebin.com/T0my7zSc - It assumes R is in least significant position
• https://mcmap.net/q/162268/-android-how-to-display-camera-preview-with-callback - It assumes R is in least significant position

I was wondering, are they making common mistake, or I have overlooked something?

Currently, my implementation is as follow.

public static void YUV_NV21_TO_RGB(int[] argb, byte[] yuv, int width, int height) {
    final int frameSize = width * height;

    final int ii = 0;
    final int ij = 0;
    final int di = +1;
    final int dj = +1;

    int a = 0;
    for (int i = 0, ci = ii; i < height; ++i, ci += di) {
        for (int j = 0, cj = ij; j < width; ++j, cj += dj) {
            int y = (0xff & ((int) yuv[ci * width + cj]));
            int v = (0xff & ((int) yuv[frameSize + (ci >> 1) * width + (cj & ~1) + 0]));
            int u = (0xff & ((int) yuv[frameSize + (ci >> 1) * width + (cj & ~1) + 1]));
            y = y < 16 ? 16 : y;

            int r = (int) (1.164f * (y - 16) + 1.596f * (v - 128));
            int g = (int) (1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
            int b = (int) (1.164f * (y - 16) + 2.018f * (u - 128));

            r = r < 0 ? 0 : (r > 255 ? 255 : r);
            g = g < 0 ? 0 : (g > 255 ? 255 : g);
            b = b < 0 ? 0 : (b > 255 ? 255 : b);

            argb[a++] = 0xff000000 | (r << 16) | (g << 8) | b;
        }
    }
}

Allhallows

For me your code works perfectly, actually is the only one that doesn't alter any of the picture properties (like brightness or colors). – Dicrotic

in many android functions that take a color argument, or return a color as int, they return with R being most significant so its 0xAARRGGBB, however in the actual memory layout of an android bitmap when you use JNI and access in C/C++ bytes directly, it's reversed with R being the first byte in memory and you get 0xAABBGGRR. – Istic

Answer 1

First of all, I am not super experienced with image encoding (has some limited exposure to this about a year ago). So, take my answer with grain of salt.

However, I believe you are right. I think in their code both a) V and U are flipped b) R and B are flipped

I have a feeling that when both of these things are flipped, it will produce the same result as if they arent' flipped. That's the reason why you can find wrong code in many places (originally, somebody got it wrong and after it was copied all over the places, because the resulting code works (however, variables named incorrectly)).

Here is another example of code (which works the same as yours): http://www.41post.com/3470/programming/android-retrieving-the-camera-preview-as-a-pixel-array

Badmouth

Answer 2

Terms like "most significant position" are ambiguous, because it depends on the endian of the machine.

When all data types are 8 bits, there is an easy unambiguous specification: byte order. For example, unsigned char rgba[4]; would have the data stored as rgba[0] = r; rgba[1] = g; rgba[2] = b; rgba[3] = a;

or {r, g, b, a } regardless of the endianness of the processor.

If instead you did

int32 color = (r << 24) | (g << 16) | (b << 8) | (a << 0);

you would get { r, g, b, a } on a big-endian system, and { a, r, g, b } on a little-endian system. Do you work on systems that have heterogeneous processors? Like maybe you have a CPU and a GPU? How do they know which endian the other is using? You are much better off defining the byte ordering.

Leclerc

Read the specification the question links to: little-endian byte order is specified. – Ireneirenic

Also, the question is tagged as Android, and all supported versions of Android use little-endian hardware. – Ireneirenic