Python OpenCV putText() show (non-ascii, unicode, utf) character symbols

Question

I am creating a sign language detection application in Armenian. However, the Armenian letters don't show when I am signing but do show in my console/terminal. What am I doing wrong with my code that is not working. Let me know how can I change or implement to show the letter on the screen. Currently any non-ascii characters display as ????? or boxes instead of the actual character or symbol like this

How do I show non-ascii characters/text instead of ?? using OpenCV's putText() method?

PS: I even try implementing using PIL but no luck. Thanks for your help in advance.

Also, when I use unicode_font instead of cv2.QT_FONT_NORMAL I get an error when I sign as:

Argument 'fontFace' is required to be an integer

Here is my code:

dict_letter = {0: 'Ա', 1: 'Բ', 2: 'Գ', 3: 'Դ', 4: 'Ե', 5: 'Զ', 6: 'Է', 7: 'Ը', 8: 'Թ', 9: 'Ժ', 10: 'Ի', 11: 'Լ', 12: 'Խ', 13: 'Ծ', 14: 'Կ', 15: 'Հ', 16: 'Ձ', 17: 'Ղ', 18: 'Ճ', 19: 'Մ', 20: 'Յ', 21: 'Ն', 22: 'Շ', 23: 'Ո', 24: 'Չ', 25: 'Պ', 26: 'Ջ', 27: 'Ռ', 28: 'Ս', 29: 'Վ', 30: 'Տ', 31: 'Ր', 32: 'Ց', 33: 'Փ', 34: 'Ք', 35: 'Ֆ'}

font_size=36
unicode_font = ImageFont.truetype("/NEW/NotoSansArmenian-VariableFont_wdth,wght.ttf", font_size)
img = cv2.putText(image_hand, f"{dict_letter[pred]}", (x_square, y_square - 5), cv2.QT_FONT_NORMAL, 0.6, (0,0,0), 2)

Updated code:

      image = np.zeros((100, 950, 3), dtype=np.uint8)
                image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
                pil_image = Image.fromarray(image)

               

                 #using mac 
                font = ImageFont.truetype("/System/Library/Fonts/Supplemental/Arial.ttf", 35)
                draw = ImageDraw.Draw(pil_image)
                draw.text((30, 30), dict_letter[pred], font=font)

 
                image = np.asarray(pil_image)
                image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
                cv2.imshow('image', image)
                cv2.waitKey() 

Baillieu

What 's if you remove this img = and see what happen? – Mignonne

Shoud this do cv2.putText(image_hand, f"{dict_letter[pred]}",(x_square, y_square - 5), unicode_font, (0,0,0), 2)? – Mignonne

Does this answer your question? cv2.putText not showing Armenian letters on screen in python OpenCV – Sukey

no............. – Baillieu

@toyotaSupra with that code, I get an error: > - Argument 'fontFace' is required to be an integer – Baillieu

Answer 1

To my knowledge, cv2.putText() only supports a small ascii subset of characters and not unicode, utf, or non-ascii characters. The short answer is that there's no built-in OpenCV functionality to display unicode characters/text symbols onto an image. But there's a workaround! We can convert the image from OpenCV to PIL, draw the non-ascii text onto the image, and then convert back to OpenCV. It's important to remember that OpenCV uses BGR while PIL uses RGB so we need to convert from BGR to RGB (OpenCV -> PIL) and RGB back to BGR (PIL -> OpenCV).

---

The idea is to create a blank mask using Numpy (or any loaded image), convert from OpenCV to PIL, draw non-ascii text onto the image, and then convert from PIL back to OpenCV. Here's a working example displaying each of the Armenian letters:

from PIL import ImageFont, ImageDraw, Image
import cv2
import numpy as np

# Create black mask using Numpy and convert from BGR (OpenCV) to RGB (PIL)
# image = cv2.imread('1.png') # If you were using an actual image
image = np.zeros((100, 950, 3), dtype=np.uint8)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
pil_image = Image.fromarray(image)

# Draw non-ascii text onto image
font = ImageFont.truetype("C:\Windows\Fonts\\arial.ttf", 35)
draw = ImageDraw.Draw(pil_image)
draw.text((30, 30), "ԱԲԳԴԵԶԷԸԹԺԻԼԽԾԿՀՁՂՃՄՅՆՇՈՉՊՋՌՍՎՏՐՑՓՔՖ", font=font)

# Convert back to Numpy array and switch back from RGB to BGR
image = np.asarray(pil_image)
image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
cv2.imshow('image', image)
cv2.waitKey()

---

Displaying other languages and/or character symbols

Depending on the language/symbol to display, you may have to modify the ImageFont.truetype() font. On Windows, all of the default fonts are located in the C:\Windows\Fonts folder. Simply select the desired font depending on the symbol you are trying to display. Right click the font file and select properties to get the exact filename and extension. For example to display Chinese characters:

font = ImageFont.truetype("C:\Windows\Fonts\simsun.ttc", 35)
draw.text((30, 30), "我是你的爸爸", font=font)

For Japanese:

font = ImageFont.truetype("C:\Windows\Fonts\YuGothL.ttc", 35)
draw.text((30, 30), "桜はかっこいい", font=font)

Note: If you're using a Mac, see Loading fonts in Python/Pillow on a Mac

Ameba

Thank you! Can I do it with dict_letter[pred]} instead of hardcoding the letters? – Baillieu

I get an error: self.font = core.getfont( OSError: cannot open resource – Baillieu

@DerekWeinstein yes you can do it with the dictionary instead of hardcoding. This is just a basic minimum reproducible example to show how to make it display properly instead of question marks. You can adapt the method to use your dictionary. Ahh okay check the update, its because its currently using a relative path instead of absolute path. Try the update it should work now – Ameba

Check my updated code I did it without the hardcoding and also please check my font if I properly loaded on a Mac. – Baillieu

So with the code it crashes my python with error code: cv2.error: Unknown C++ exception from OpenCV code Assertion failed: (NSViewIsCurrentlyBuildingLayerTreeForDisplay() != currentlyBuildingLayerTree), function NSViewSetCurrentlyBuildingLayerTreeForDisplay, file NSView.m, line 13477. Illegal instruction: 4 – Baillieu

what version of opencv are using for this kind of code? maybe thats the issue. – Baillieu

I don't have a Mac so I can't confirm if you loaded it correctly. I'm not sure how you're getting that error. Is that error from the code snippet or your own code? Python v3.7.9, opencv-python v4.4.0.46, opencv-contrib-python v3.4.2.17, Pillow v8.0.1 – Ameba

why is the code crashing based on the error? – Baillieu

I don't know where that error is coming from it doesnt look like its coming from part of the snippet. It's probably some section from your own code. Try do it with a single letter first before you add in the dictionary – Ameba

Ok. Also, is the code snippet will show all letters or it will go based on a sign language hand gesture and then it will show.? – Baillieu

and did this work on windows for you? – Baillieu

Let us continue this discussion in chat. – Baillieu

@DerekWeinstein I am displaying Korean text after recognition but nonthing display on matplotlib. – Sergias