Image comparison to identity and map identical pixels

Question

I'm building this for iOS using Swift — either via CoreImage or GPUImage, but if I can build it in Python or Node/JavaScript, that'd work too. Feel free to answer abstractly, or in a different language entirely — I'll accept any answer that roughly describes how I might go about accomplishing this.

Consider the following two "images" (I've fabricated two 3x3-pixel grids to represent two images, each 3x3 pixels for a total of 9 pixels).

Let's assume I process the original image (left) with a shader that changes the color of some, but not all of the pixels. The resulting image on the right is the same, but for 3 pixels — #2, #3, and #6:

I'm trying to find a means of comparing all of the pixels in both images and logging the x,y position of pixels that haven't changed during the filter process. In this case, when comparing the left to right, I'd need to know that #1, #4, #5, #7, #8, and #9 remained unchanged.

Dylane

Wow. Up-voted. I understand GPU cycles versus CPU cycles, but this? I'm guessing you are asking something that won't happen in real-time, near real-time, maybe even in acceptable time. It really depends on how many pixels you are comparing. EDIT: if all you want is to compare a 3x3 pixel over two image extents, use a general CI kernel. For iOS it'll perform best, and depending on what you want for pixel output it'll work. – Gynecic

64³. No, definitely won't happen in real-time. The backstory here is that I'm using a lookup-table to process images, and the source LUTs are ~1MB in size. That's huge. I'm investigating whether I can significantly compress them without losing data. If I can eliminate pixels in the LUT that match the source file, I'm hoping they'll end up a lot smaller. – Dylane

You might have this work. Again, if iOS, you probably want CoreImage for performance and a general CI kernel for code. I'd suggest doing everything in a single call but if you are compressing maybe not. Instead, maybe a mask on the first pass and then compress on the second? Good luck. – Gynecic

BTW, this isn't really Swift - OBJ-C or Swift (or anything else for that matter) doesn't matter. Again, good luck! – Gynecic

To compress I'll just black out all the duplicate pixels outside of the image outside of compile or runtime. Then I'd do a single pass with a mask inside CI kernel which makes sense. The challenge is how I'm going to do that 😊 And yes, I realize it's language-agnostic, hence my disclaimer above. – Dylane

"In this case, when comparing the left to right, I'd need to know that #1, #4, #5, #7, #8, and #9 remained unchanged" - if you're passing that data to a subsequent filter, could you use CIDifferenceBlendMode with your two images as inputs and use its output as a mask to the next filter? – Pilcomayo

Ooo nice idea @SimonGladman – Dylane

Answer 1

Assuming your images before and after are the same size all you need to do is loop through each pixel and compare them which you can do with a pointer. I certainly don't claim this is the fastest method but it should work (note you can compare all 32 bits at once with a UInt32 pointer, but I am doing it byte wise just to illustrate where the RGBA values are if you need them). Also note that because of the fact that Quartz was written for Mac and it uses Cartesian coordinates and iOS and UIKit do not, its possible your data is upside down (mirrored around the X-axis). You will have to check; it depends on how the internal bitmap is being represented.

  func difference(leftImage: UIImage, rightImage: UIImage) {
      let width = Int(leftImage.size.width)
      let height = Int(leftImage.size.height)
      guard leftImage.size == rightImage.size else {
          return
      }
      if let cfData1:CFData = leftImage.cgImage?.dataProvider?.data,
         let l = CFDataGetBytePtr(cfData1),
         let cfData2:CFData = rightImage.cgImage?.dataProvider?.data,
         let r = CFDataGetBytePtr(cfData2) {
          let bytesPerpixel = 4
          let firstPixel = 0
          let lastPixel = (width * height - 1) * bytesPerpixel
          let range = stride(from: firstPixel, through: lastPixel, by: bytesPerpixel)
          for pixelAddress in range {
              if l.advanced(by: pixelAddress).pointee != r.advanced(by: pixelAddress).pointee ||     //Red
                 l.advanced(by: pixelAddress + 1).pointee != r.advanced(by: pixelAddress + 1).pointee || //Green
                 l.advanced(by: pixelAddress + 2).pointee != r.advanced(by: pixelAddress + 2).pointee || //Blue
                 l.advanced(by: pixelAddress + 3).pointee != r.advanced(by: pixelAddress + 3).pointee  {  //Alpha
                  print(pixelAddress)
                  // do stuff here
              }
          }
      }
  }

If you need a faster method write a shader that will delta each pixel and write the result out to a texture. Any pixels that are not clear black (i.e. 0,0,0,0) in the output are different between the images. Shaders are not my area of expertise so I will leave it to someone else to write. Also on some architectures its expensive to read back form graphics memory so you will have to test and see if this is really better than doing it in main memory (may also depend on image size because you have to amortize the setup cost for the textures and shaders).

Ajax

👏👏👏👏👏 Fantastic! – Dylane

Thank you for having shared this, this is really interesting – Ozellaozen

Answer 2

I use another option, a slightly modified Facebook version.

The original code here

func compareWithImage(_ referenceImage: UIImage, tolerance: CGFloat = 0) -> Bool {
    guard size.equalTo(referenceImage.size) else {
        return false
    }
    guard let cgImage = cgImage, let referenceCGImage = referenceImage.cgImage else {
        return false
    }
    let minBytesPerRow = min(cgImage.bytesPerRow, referenceCGImage.bytesPerRow)
    let referenceImageSizeBytes = Int(referenceImage.size.height) * minBytesPerRow
    let imagePixelsData = UnsafeMutablePointer<Pixel>.allocate(capacity: cgImage.width * cgImage.height)
    let referenceImagePixelsData = UnsafeMutablePointer<Pixel>.allocate(capacity: cgImage.width * cgImage.height)

    let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue & CGBitmapInfo.alphaInfoMask.rawValue)

    guard let colorSpace = cgImage.colorSpace, let referenceColorSpace = referenceCGImage.colorSpace else { return false }

    guard let imageContext = CGContext(data: imagePixelsData, width: cgImage.width, height: cgImage.height, bitsPerComponent: cgImage.bitsPerComponent, bytesPerRow: minBytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue) else { return false }
    guard let referenceImageContext = CGContext(data: referenceImagePixelsData, width: referenceCGImage.width, height: referenceCGImage.height, bitsPerComponent: referenceCGImage.bitsPerComponent, bytesPerRow: minBytesPerRow, space: referenceColorSpace, bitmapInfo: bitmapInfo.rawValue) else { return false }

    imageContext.draw(cgImage, in: CGRect(x: 0, y: 0, width: size.width, height: size.height))
    referenceImageContext.draw(referenceCGImage, in: CGRect(x: 0, y: 0, width: referenceImage.size.width, height: referenceImage.size.height))

    var imageEqual = true

    // Do a fast compare if we can
    if tolerance == 0 {
        imageEqual = memcmp(imagePixelsData, referenceImagePixelsData, referenceImageSizeBytes) == 0
    } else {
        // Go through each pixel in turn and see if it is different
        let pixelCount = referenceCGImage.width * referenceCGImage.height

        let imagePixels = UnsafeMutableBufferPointer<Pixel>(start: imagePixelsData, count: cgImage.width * cgImage.height)
        let referenceImagePixels = UnsafeMutableBufferPointer<Pixel>(start: referenceImagePixelsData, count: referenceCGImage.width * referenceCGImage.height)

        var numDiffPixels = 0
        for i in 0..<pixelCount {
            // If this pixel is different, increment the pixel diff count and see
            // if we have hit our limit.
            let p1 = imagePixels[i]
            let p2 = referenceImagePixels[i]

            if p1.value != p2.value {
                numDiffPixels += 1

                let percents = CGFloat(numDiffPixels) / CGFloat(pixelCount)
                if percents > tolerance {
                    imageEqual = false
                    break
                }
            }
        }
    }

    free(imagePixelsData)
    free(referenceImagePixelsData)

    return imageEqual
}

struct Pixel {

    var value: UInt32

    var red: UInt8 {
        get { return UInt8(value & 0xFF) }
        set { value = UInt32(newValue) | (value & 0xFFFFFF00) }
    }

    var green: UInt8 {
        get { return UInt8((value >> 8) & 0xFF) }
        set { value = (UInt32(newValue) << 8) | (value & 0xFFFF00FF) }
    }

    var blue: UInt8 {
        get { return UInt8((value >> 16) & 0xFF) }
        set { value = (UInt32(newValue) << 16) | (value & 0xFF00FFFF) }
    }

    var alpha: UInt8 {
        get { return UInt8((value >> 24) & 0xFF) }
        set { value = (UInt32(newValue) << 24) | (value & 0x00FFFFFF) }
    }

}

Sunken