How to overlay a png on a webcam feed?

Question

I have a png image of shape(480,640,4) and webcam frame of shape(480,640,3). I would like to overlay the png entirely over the webcam feed but I'm getting the following error:

added_image = cv2.addWeighted(frame,0.4,png,0.1,0)

The error:

cv2.error: OpenCV(4.2.0) /io/opencv/modules/core/src/arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'arithm_op'

Is the issue because of the channel difference. Can someone help me solve this issue? Thanks in advance!

Joan

Answer 1

If you use addWeighted() you will get the same constant fraction of your overlay image everywhere on your background image. It seems unlikely that is what you want because your input image has 4-channels, of which one is presumably the alpha channel. So normally you would expect the background image to show through differently according to the alpha channel.

Taking this as the background (webcam) image with shape (500,300,3):

And this as the overlay image with transparency and shape (500,300,4):

import cv2
import numpy as np

# Read background and overlay images
bg = cv2.imread('buckinghampalace.jpg')
ol = cv2.imread('overlay.png', cv2.IMREAD_UNCHANGED) 

# Make a result image that is either the background image or the overlay image depending on the alpha channel being < 128
res = np.where((ol[...,3]<128)[...,None], bg, ol[...,0:3])

cv2.imwrite('result.jpg', res)

Crinoline

Can you explain me a bit on the < 128 calculation – Joan

Channels 0,1,2 are the RGB values in the overlay image, whereas channel 3 is the alpha (transparency) layer. Anywhere that layer is 255, the overlay image is 100% transparent and the background (webcam) will show through, Anywhere that layer is 0, the overlay image is 100% opaque and will fully conceal the background (webcam) image. So I said I'll take 128 as the 50% transparent point and anything less than that I put the overlay RGB values in the output image and anywhere more than that I put the webcam RGB values in the output image. – Crinoline

Answer 2

This is because of that your images have different number of channels( one is 4 channel the other 3 channel). If you check the documentation about addWeighted(). It says about first source image(src1) and second source image(src2):

src1: first input array.

src2: second input array of the same size and channel number as src1.

Exudate

Yes you are right!, I solved it by converting rgba to rgb before giving it to addweight(). Now its worling fine – Joan

Answer 3

It's because the images don't have the same dimension. You must convert the PNG image from BGRA to BGR. Then the error should be gone.

png = cv2.cvtColor(png, cv2.COLOR_BGRA2BGR)
added_image = cv2.addWeighted(frame,0.4,png,0.1,0)

Flag