How to remove hidden marks from images using python opencv?

Question

I wanted to work on a small project to challenge my computer vision and image processing skills. I came across a project where I want to remove the hidden marks from the image. Hidden here refers to the watermarks that are not easily visible in rgb space but when you convert into hsv or some other space the marks become visible.

Here's one example:

BGR SPACE:

HSV SPACE:

I've tried different ways but was able to implement a solution that would remove those watermarks from the image. I am posting this question here to get different ideas to tackle this problem.

What I have tried:

I have tried various approaches but none of them worked, sharing the code might not help. It is not necessary to provide code for it, a pseudo code, idea or any lead would be appreciated.

1. I noticed that the hidden marks are all the colors similar to RGB(90,94,105). And when I showed R, G, and B separately I noticed that the watermarks were only visible in B channel. I thought that if adjust/remove the marks in B channel and merge the image again, may be I could get better results.

Code:

b,g,r = cv2.split(img)
b = b//2;
r = cv2.merge((r,g,b))
cv2.imshow("image",r)

Problems: This doesn't does solve the problem, it did make the colors little dimmer but the image colors were also disturbed.

2. I tried playing around with B channel to see if could accomplish something.

3. I also noticed that if we convert the image to LUV space then the marks are visible in V space.

Lymph

What "various approaches" are you speaking of? That isn't enough to go on. – Altercate

Let me update the question to elaborate. In the mean time can you share your thoughts about the problem – Lymph

Interesting problem. I'll look at this soon. I'm not able to currently. – Altercate

Please edit a minimal reproducible example of your best solution so far into your question. – Goulette

that's the best I've got to. Moreover, the reason to put bounty on this question was to see different approaches and not necessarily an extended version of what I did. I hope the question is clear enough to proceed further. The best I can do in addition is to provide more sample – Lymph

Btw if you check RGB channels, the numbers only visible on Blue channel. I tried to go through on this but couldnt achieve – Lambskin

I also noticed that the marks are visible only in the blue channel, maybe this lead could be extended by others – Lymph

@Altercate did you think of any solution. Any leads would be really appreciated. – Lymph

My apologies but I haven't. That's why I haven't made an answer. – Altercate

@Altercate , i have added answer. I hope you would be able to make possible suggestions or improvements to that approach – Lymph

Answer 1

I didn't find any answer that completely solved the question. I appreciate everyone's effort though (Thank you). I did something on my own and would like to share. It results in little quality loss (a little bluish blurriness) but successfully removes the watermarks. The solution is very simple but took time to analyze the image.

I WOULD BE VERY GLAD IF SOMEONE CAN EXTEND THIS APPROACH AND COME UP WITH SOMETHING EVEN BETTER

I observed that the watermarks were only visible in B space (out of RGB) and there were no traces of watermarks in R and G space.

B space:

I also red somewhere that blue light contributes little to the overall image compared to R and G channel so here's what I decided to do.

Blur the B channel by a large enough amount to remove traces of those patterns. Here's how the B channel would appear afterwards:

Finally, merge the image with the new B channel, previous R and previous G channel. Here's how the RGB channel would appear afterwards:

The advantage of using approach is that the traces are gone.

The only disadvantage is that the bluish and purplish colors appear at the black edges and the image is a little bluish in general.

My Code:

import cv2
from matplotlib import pyplot as plt
import numpy as np
img = cv2.imread("img.png")
b, g, r = cv2.split(img) # split into B,G,R spaces 
b = cv2.GaussianBlur(b, None, 8)
plt.imshow(cv2.merge((r,g,b)), cmap='gray')

Lymph

Answer 2

This might be a possible approach. The underlying idea is that there are edges visible in the HSV channel that are not present in the original image. Here are the H, S and V channels side-by-side:

So if we find the edges in the original image and the edges in the HSV image and difference them, the watermarking should show up. That can then be used as a mask to do in-painting in the original image with OpenCV inpaint.

I am just using ImageMagick here in Terminal, but it could all be done equally with OpenCV, PIL or scikit-image:

# Detect edges visible in original image and auto-level
convert watermarked.png -colorspace gray -auto-level -canny 0x1+1%+3% -auto-level  RGB-edges.png

# Find visible edges in H, S and V colourspace, generate mean across all three and auto-level
convert watermarked.png -colorspace hsv -separate -canny 0x1+1%+3% -evaluate-sequence mean -auto-level HSV-edges.png

# Find changemask between the two sets of edges
convert RGB-edges.png HSV-edges.png -compose changemask -composite result.png

The idea is that the watermarking is now identified in black, so use the black areas (maybe morphologically closed) as a mask in OpenCV to inpaint - see link above.

Ameliaamelie

I also tried to use this approach but the noise during finding contours doesnt let the get good results. In your results also the noise exist – Lambskin

Answer 3

Here is a slight variation and extension of your processing in Python/OpenCV.

The main difference is that I use the median rather than a blurring and that I try to extract the black lines and impose them on the median before recombining.

Input:

import cv2
import numpy as np

# read image
img = cv2.imread("cartoon_hidden_marks.png")

# separate channels
b,g,r = cv2.split(img)

# median filter blue
median = cv2.medianBlur(b, 21)

# threshold blue image to extract black lines
thresh = cv2.threshold(b, 20, 255, cv2.THRESH_BINARY)[1]

# apply thresh to median
b_new = cv2.bitwise_and(median, thresh)

# combine b_new, g, b
img_new = cv2.merge([b_new,g,r])

# write results to disk
cv2.imwrite("cartoon_hidden_marks_median.jpg", median)
cv2.imwrite("cartoon_hidden_marks_thresh.jpg", thresh)
cv2.imwrite("cartoon_hidden_marks_new_blue.jpg", b_new)
cv2.imwrite("cartoon_hidden_marks_result.png", img_new)

# display it
cv2.imshow("median", median)
cv2.imshow("thresh", thresh)
cv2.imshow("b_new", b_new)
cv2.imshow("img_new", img_new)
cv2.waitKey(0)

Blue channel median:

Blue channel threshold (for black lines):

New blue channel:

Result:

Many of the erroneous blue lines are now black, but not all. Increasing the threshold would have gotten more black lines, but then the hidden marks would have appeared again in part.

Hoeve

Bluishness is removed but most of the white regions are converted to light brown. – Lymph

Answer 4

If you have managed to isolate the watermarks in any channel, you should be able to threshold it and create a binary mask. Then you could use inpainting to fill the gaps with something like:

    clean_image = cv2.inpaint(marked_image, mask_of_marks, 3, cv2.INPAINT_TELEA)

Scorch

The OP hasn't isolated them out. That's the point of the question. – Altercate

Answer 5

Another trivial solution in Python/OpenCV is simply to replace the green channel for the blue channel, since most of the green channel is about the same intensity distribution as that of the blue channel.

Input:

import cv2
import numpy as np

# read image
img = cv2.imread("cartoon_hidden_marks.png")

# separate channels
b,g,r = cv2.split(img)

# combine replacing b with g
img_new = cv2.merge([g,g,r])

# write results to disk
cv2.imwrite("cartoon_hidden_marks_result2.png", img_new)

# display it
cv2.imshow("result", img_new)
cv2.waitKey(0)

Result:

The issue is that the coat and the green tree are slightly different color and texture.

One might try modifying a copy of the green channel image to have the mean and standard-deviation as the blue channel to fix the coat issue. For the green tree, it is outside the region of the watermark, so one could mask that using inRange for the green tree color and then replace the blue channel image's tree in the copy of the green channel. Then recombine the modified green channel in place of the blue channel.

Hoeve