Superimpose heatmap on a base image OpenCV Python

Question

Please look at this github page. I want to generate heat maps in this way using Python PIL,open cv or matplotlib library. Can somebody help me figure it out?

I could create a heat map for my network at the same size as the input, but I am not able superimpose them. The heatmap shape is (800,800) and the base image shape is (800,800,3)

Debose

did you get any solution ? i do have same kind of requirements – Petronia

Answer 1

Updated Answer -- 29th April, 2022.

After the repeated comments I have decided to update this post with a better visualization.

Consider the following image:

img = cv2.imread('image_path')

I obtained a binary image after performing binary threshold on the a-channel of the LAB converted image:

lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
a_component = lab[:,:,1]
th = cv2.threshold(a_component,140,255,cv2.THRESH_BINARY)[1]

Applying Gaussian blur:

blur = cv2.GaussianBlur(th,(13,13), 11)

The resulting heatmap:

heatmap_img = cv2.applyColorMap(blur, cv2.COLORMAP_JET)

Finally, superimposing the heatmap over the original image:

super_imposed_img = cv2.addWeighted(heatmap_img, 0.5, img, 0.5, 0)

Note: You can vary the weight parameters in the function cv2.addWeighted and observe the differences.

Improvvisatore

I mean the second image. please correct it! heated_img and fin look same. – Debose

Your overall method is correct. But second and third image look the same, that's it! Change the picture of second. – Debose

Let us continue this discussion in chat. – Debose

Answer works well. However it's odd to create a direct heatmap of an image and overlay this on the image. I think that's what confused @curio1729. Typically the heatmap overlaid is a result of estimation/prediction/other done on the image. – Galoshes

I agree. This would've been more clear if you dramatically blurred the input. I understand your heatmap is just an example, but because it is exactly the same underlying data is the original image, it doesn't look like what folks are expecting. – Clearance

Answer 2

My code starts from a heatmap matrix (224,224) called cam, which is applied to the original image called frame, via opencv;

and it seems to work pretty well:

import numpy as np
from cv2 import cv2
from skimage import exposure 
...

capture = cv2.VideoCapture(...)
while True:
    ret, frame = capture.read()

    if ret:
        #resize original frame
        frame = cv2.resize(frame, (224, 224)) 

        #get color map
        cam = getMap(frame)
        map_img = exposure.rescale_intensity(cam, out_range=(0, 255))
        map_img = np.uint8(map_img)
        heatmap_img = cv2.applyColorMap(map_img, cv2.COLORMAP_JET)

        #merge map and frame
        fin = cv2.addWeighted(heatmap_img, 0.5, frame, 0.5, 0)

        #show result
        cv2.imshow('frame', fin)

the getMap() function gets the headmap given the frame;

I found some interesting free videos about this topic:

https://www.youtube.com/watch?v=vTY58-51XZA&t=14s

https://www.youtube.com/watch?v=4v9usdvGU50&t=208s

Sixtasixteen

Answer 3

I had some problems with grayscale images at this line

super_imposed_img = cv2.addWeighted(heatmap_img, 0.5, img, 0.5, 0)

but this one worked for me

plt.imshow(binary_classification_result * 0.99 + original_gray_image * 0.01)

Propagandist

Answer 4

On the page you shared, it is done through a pre-trained model. You can use backbones like ResNet:

Load the backbone:

resnet_50 = tf.keras.applications.ResNet50(input_shape=(224, 224, 3),
                                           include_top=False,
                                           weights='imagenet')

Preprocess the image so that it matches resnet's default

img = cv2.imread("/content/your_image.jpg")[:,:,::-1]
img = cv2.resize(image, (224, 224))
ax = plt.imshow(img)

def preprocess(img):
  # use the pre processing function of ResNet50 
  img = preprocess_input(img)
  
  #expand the dimension
  return np.expand_dims(img, 0)

input_image = preprocess(img)

Applying this paper's suggestions:

def postprocess_activations(activations):

  output = np.abs(activations)
  output = np.sum(output, axis = -1).squeeze()

  #resize and convert to image 
  output = cv2.resize(output, (224, 224))
  output /= output.max()
  output *= 255
  return 255 - output.astype('uint8')

Generate and plot heatmaps:

def apply_heatmap(weights, img):

   #generate heat maps 
   heatmap = cv2.applyColorMap(weights, cv2.COLORMAP_JET)
   heatmap = cv2.addWeighted(heatmap, 0.7, img, 0.3, 0)
return heatmap

Plot:

 def plot_heatmaps(rng):
  level_maps = None
  
  #given a range of indices generate the heat maps 
  for i in rng:
    activations = get_activations_at(input_image, i)
    weights = postprocess_activations(activations)
    heatmap = apply_heatmap(weights, img)
    if level_maps is None:
      level_maps = heatmap
    else:
      level_maps = np.concatenate([level_maps, heatmap], axis = 1)
  plt.figure(figsize=(15, 15))
  plt.axis('off')
  ax = plt.imshow(level_maps)
plot_heatmaps(range(164, 169))

Wivestad