How to merge UIImages while using transform (scale, rotation and translation)?

Question

I'm trying to replicate a function of Instagram where you have your picture and you can add stickers (others images) and then save it.

So on the UIImageView that holds my picture, I add the sticker (another UIImageView) to it as a subview, positioned at the center of the parent UIImageView.

To move the sticker around the picture, I do it using a CGAffineTransform (I don't move the center of the UIImageView). I also apply a CGAffineTransform for rotating and scaling the sticker.

To save the picture with the stickers, I use a CGContext as following:

    extension UIImage {
        func merge2(in rect: CGRect, with imageTuples: [(image: UIImage, viewSize: CGSize, transform: CGAffineTransform)]) -> UIImage? {
            UIGraphicsBeginImageContextWithOptions(size, false, UIScreen.main.scale)

            guard let context = UIGraphicsGetCurrentContext() else { return nil }

            draw(in: CGRect(size: size), blendMode: .normal, alpha: 1)

            // Those multiplicators are used to properly scale the transform of each sub image as the parent image (self) might be bigger than its view bounds, same goes for the subviews
            let xMultiplicator = size.width / rect.width
            let yMultiplicator = size.height / rect.height

            for imageTuple in imageTuples {
                let size = CGSize(width: imageTuple.viewSize.width * xMultiplicator, height: imageTuple.viewSize.height * yMultiplicator)
                let center = CGPoint(x: self.size.width / 2, y: self.size.height / 2)
                let areaRect = CGRect(center: center, size: size)

                context.saveGState()

                let transform = imageTuple.transform
                context.translateBy(x: center.x, y: center.y)
                context.concatenate(transform)
                context.translateBy(x: -center.x, y: -center.y)

// EDITED CODE
                context.setBlendMode(.color)
                UIColor.subPink.setFill()
                context.fill(areaRect)
// EDITED CODE
                imageTuple.image.draw(in: areaRect, blendMode: .normal, alpha: 1)

                context.restoreGState()
            }

            let image = UIGraphicsGetImageFromCurrentImageContext()
            UIGraphicsEndImageContext()

            return image
        }
    }

The rotation inside the transform is taken into account. The scaling inside the transform is taken into account.

But the translation inside the transform seems to no work (there is a tiny translation but it doesn't reflect the real one).

I'm obviously missing something here but can't find out what.

Any idea?

EDIT:

Here are some screenshots of how the sticker looks like on the app and what is the final image saved in the library. As you can see, the rotation and scale (the width/height ratio) of the final image are the same than the one in the app.

The UIImageView holding the UIImage has the same ratio than its image.

I also added a background when drawing the sticker to clearly see the bounds of the actual image.

No rotation or scaling:

Rotated and scaled:

EDIT 2:

Here is a test project that reproduces the behaviour described above.

Freedafreedman

Can you post a picture of desired effect you want to achieve? – Mycology

I tested this code and translation does work, so it's not clear what is going wrong in your case. Or, it's not clear what your expectation is vs the result. – Kimmy

Sorry for the late reply. @Kuba I updated the question with screenshots. – Freedafreedman

@KenBoreham please have a look on the screenshots – Freedafreedman

Answer 1

The problem is you have multiple geometries (coordinate systems) and scale factors and reference points in play, and it's hard to keep them straight. You have the root view's geometry, in which the frames of the image view and the sticker view are defined, and then you have the geometry of the graphics context, and you don't make them match. The image view's origin is not at the origin of its superview's geometry, because you constrained it to the safe areas, and I'm not sure you're properly compensating for that offset. You try to deal with the scaling of the image in the image view by adjusting the sticker size when drawing the sticker. You don't properly compensate for the fact that you have both the sticker's center property and its transform affecting its pose (location / scale /rotation).

Let's simplify.

First, let's introduce a “canvas view” as the superview of the image view. The canvas view can be laid out however you want with respect to the safe areas. We'll constrain the image view to fill the canvas view, so the image view's origin will be .zero.

Next, we'll set the sticker view's layer.anchorPoint to .zero. This makes the view's transform operate relative to the top-left corner of the sticker view, instead of its center. It also makes view.layer.position (which is the same as view.center) control the position of the top-left corner of the view, instead of controlling the position of the center of the view. We want these changes because they match how Core Graphics draws the sticker image in areaRect when we merge the images.

We'll also set view.layer.position to .zero. This simplifies how we compute where to draw the sticker image when we merge the images.

private func makeStickerView(with image: UIImage, center: CGPoint) -> UIImageView {
    let heightOnWidthRatio = image.size.height / image.size.width
    let imageWidth: CGFloat = 150

    //      let newStickerImageView = UIImageView(frame: CGRect(origin: .zero, size: CGSize(width: imageWidth, height: imageWidth * heightOnWidthRatio)))
    let view = UIImageView(frame: CGRect(x: 0, y: 0, width: imageWidth, height: imageWidth * heightOnWidthRatio))
    view.image = image
    view.clipsToBounds = true
    view.contentMode = .scaleAspectFit
    view.isUserInteractionEnabled = true
    view.backgroundColor = UIColor.red.withAlphaComponent(0.7)
    view.layer.anchorPoint = .zero
    view.layer.position = .zero
    return view
}

This means we need to position the sticker entirely using its transform, so we want to initialize the transform to center the sticker:

@IBAction func resetPose(_ sender: Any) {
    let center = CGPoint(x: canvasView.bounds.midX, y: canvasView.bounds.midY)
    let size = stickerView.bounds.size
    stickerView.transform = .init(translationX: center.x - size.width / 2, y: center.y - size.height / 2)
}

Because of these changes, we have to handle pinches and rotates in a more complex way. We'll use a helper method to manage the complexity:

extension CGAffineTransform {
    func around(_ locus: CGPoint, do body: (CGAffineTransform) -> (CGAffineTransform)) -> CGAffineTransform {
        var transform = self.translatedBy(x: locus.x, y: locus.y)
        transform = body(transform)
        transform = transform.translatedBy(x: -locus.x, y: -locus.y)
        return transform
    }
}

Then we handle pinch and rotate like this:

@objc private func stickerDidPinch(pincher: UIPinchGestureRecognizer) {
    guard let stickerView = pincher.view else { return }
    stickerView.transform = stickerView.transform.around(pincher.location(in: stickerView), do: { $0.scaledBy(x: pincher.scale, y: pincher.scale) })
    pincher.scale = 1
}

@objc private func stickerDidRotate(rotater: UIRotationGestureRecognizer) {
    guard let stickerView = rotater.view else { return }
    stickerView.transform = stickerView.transform.around(rotater.location(in: stickerView), do: { $0.rotated(by: rotater.rotation) })
    rotater.rotation = 0
}

This also makes scaling and rotating work better than before. In your code, scaling and rotating always happen around the center of the view. With this code, they happen around the center point between the user's fingers, which feels more natural.

Finally, to merge the images, we'll start by scaling the graphics context's geometry the same way imageView scaled its image, because the sticker transform is relative to the imageView size, not the image size. Since we position the sticker entirely using the transform now, and since we've set the image view and sticker view origins to .zero, we don't have to make any adjustments for weird origins.

extension UIImage {

    func merge(in viewSize: CGSize, with imageTuples: [(image: UIImage, viewSize: CGSize, transform: CGAffineTransform)]) -> UIImage? {
        UIGraphicsBeginImageContextWithOptions(size, false, UIScreen.main.scale)

        print("scale : \(UIScreen.main.scale)")
        print("size : \(size)")
        print("--------------------------------------")

        guard let context = UIGraphicsGetCurrentContext() else { return nil }

        // Scale the context geometry to match the size of the image view that displayed me, because that's what all the transforms are relative to.
        context.scaleBy(x: size.width / viewSize.width, y: size.height / viewSize.height)

        draw(in: CGRect(origin: .zero, size: viewSize), blendMode: .normal, alpha: 1)

        for imageTuple in imageTuples {
            let areaRect = CGRect(origin: .zero, size: imageTuple.viewSize)

            context.saveGState()
            context.concatenate(imageTuple.transform)

            context.setBlendMode(.color)
            UIColor.purple.withAlphaComponent(0.5).setFill()
            context.fill(areaRect)

            imageTuple.image.draw(in: areaRect, blendMode: .normal, alpha: 1)

            context.restoreGState()
        }

        let image = UIGraphicsGetImageFromCurrentImageContext()
        UIGraphicsEndImageContext()

        return image
    }
}

You can find my fixed version of your test project here.

Salters

Hi @rob mayoff, I am using your code and it works perfectly. But since the anchor point is set to .zero, when I write a function that rotates the UIView using matrix transformation, it will rotate from topleft as center. Is their a way to fix this one? If I set anchor point to CGPoint(0.5, 0.5) it will misplace the stickerView when saved. – Cowl

The problem you are describing sounds like the entire reason I wrote the CGAffineTransform.around method in this answer. – Salters

Hey @rob mayoff , I used your sample for do it , but I've got a problem , when main image contents mode changed to .scaleAspectFit , it doesn't work well and draw this on another place – Araminta

Thank you @robmayoff you saved my day. How can I limit pan/pinch areas of top image on bottom image? – Meaghan

Implement gestureRecognizer(_:shouldReceive:) in your UIGestureRecognizerDelegate. – Salters

@robmayoff I also looking for some solution like yours. This is very good idea but I think we will lose original size of background image with this way¿ – Inglorious

@robmayoff your code is working as desired while using imageView.contentMode = scaleoFill so when I changed it .scaleAspectFit it changed image scale and merged sticker to wrong location. – Inglorious

Answer 2

I believe that CGAffineTransform properties (scale, rotate and translate) each works only on one view at a time. So, when you say you are trying to achieve all three transformation properties, I believe you have been trying on only two UIImageViews ( one for rotation and another for scaling) Add one more UIImageView for translation property.

Hope it helps.

Scopolamine