makemessages command results in html.py files and a UnicodeDecodeError

Question

When I run this command:

django-admin makemessages -l ar

It gives this error:

Traceback (most recent call last):
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\utils\encoding.py", line 65, in force_text
s = str(s, encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 3107: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Ahmed\AppData\Local\Programs\Python\Python36-32\Scripts\django-admin.exe\__main__.py", line 9, in <module>
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site- packages\django\core\management\__init__.py", line 371, in execute_from_command_line
utility.execute()
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site- packages\django\core\management\base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\base.py", line 335, in execute
output = self.handle(*args, **options)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\commands\makemessages.py", line 384, in handle
potfiles = self.build_potfiles()
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\commands\makemessages.py", line 426, in build_potfiles
self.process_files(file_list)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\commands\makemessages.py", line 522, in process_files
self.process_locale_dir(locale_dir, files)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\commands\makemessages.py", line 590, in process_locale_dir
msgs, errors, status = popen_wrapper(args)
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\core\management\utils.py", line 23, in popen_wrapper
force_text(output, stdout_encoding, strings_only=True, errors='strict'),
File "c:\users\ahmed\appdata\local\programs\python\python36-32\lib\site-packages\django\utils\encoding.py", line 69, in force_text
raise DjangoUnicodeDecodeError(s, *e.args)
django.utils.encoding.DjangoUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 3107: invalid continuation byte. You passed in 

Aside from the error, html.py files are created for each html file. For example: for index.html, an index.html.py is created with it in the template folder. These html.py files contains just 'X' and 'B' characters with some text to translate.

For example:

BBBBBBBB
BBBBB BBBBBBB
    XXXX XXXXXXXXXXXXXXXXXXXXX
        XXXX XXXXXXXXX XXXXX XXX
        XXXX XXXXXXXXXXXXXXXXXX
              XXXX XXXXXXXXXXXXX
                      XXXX XXXXXXXXXXXXXXXXXX XX
                          XXX XXXXXXXXXXXXXX gettext(u'Les appels à projet') XXXXX
                        XXXX XXXXXXXXXXXXXXXXXXXXXXXXXX
                            XXX XXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX
                                XXX XXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXX XXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXX
                                XXXXXX XXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXX
                                XXXXXX XXXXXXXXXXXXXX XXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXX
                            XXXXX
                        XXXXXX
                      XXXXXX
                      XXXX XXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXX
                        XXXX XXXXXXXXXXXXXXXXXXXXXXXXX
                          XXXXXX XXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXX
                                XXXXXXX
                                    XXXX
                                        XXXX gettext(u'Appel à projet') XXXXX
                                        XXXX gettext(u'Date de lencement') XXXXX
                                        XXXX gettext(u'Date de clature') XXXXX
                                        XXXX gettext(u'Année budjitaire') XXXXX
                                        XXXX gettext(u"N° d'apppel à projet") XXXXX
                                        XXXX gettext(u"Etat d'appel") XXXXX
                                        XXXX XXXX gettext(u'Action') XXXXX XXX
                                    XXXXX
                                XXXXXXXX
                                XXXXXXX
                                BBB BBBBBB BB BBBBBBBBBBB
                                    XXXX
                                        XXXXXXXXX
                                        XXXXXXXXX
                                        XXXXXXXXX

I already have a .po file for Ar with some text to translate, which I already created 2 months ago, and it's translated perfectly.

But when I run the makemessages command to add the new text to translate, it gives the above error.

Any ideas please?

Warmblooded

Files with .html.py or .txt.py extension are temporary and are normally removed automatically during cleanup() when makemessages runs. It seems that the exceptions prevented the cleanup of temporary files. – Embellish

Answer 1

I found the solution, I just forget some basics :p

The problem is in the templates name, I have two templates file named CritéreGlobale.html and CritéreSpecifique.html the problem is the character é in the names of files, after I edited the names to CritereGlobale.html and CritereSpecifique.html the problem solved and no more .html.py files, and the command run perfectly.

Hope that help someone else

Warmblooded

Answer 2

I had the same issue with the .html.py duplicated files. I ended up with duplicated .txt.py files too, even in my static folder!. And the solution was not utf-8 related.

My problem was that I executed django-admin makemessages -l <locale> at a project level directory. The commands makemessages and compilemessages must be executed being inside the app directory.

As a side note, don't forget to create the locale folder inside the app first, before executing the commands. Also execute the makemessages command using the locale name string (with the form ll_LL, with underscore and capital letter for country) and not the language code (with the form ll-ll, with dash and no capitals)

I hope this helps other people, as this was the only question I found in stackoverflow related to this issue.

Hitormiss

Answer 3

Instead of:

gettext(u'Les appels à projet')

Try:

gettext('Les appels à projet')

Hope it helps.

Crenate

I dont use the gettext(u'Les appels à projet') in my templats, it generated automatically in the .html.py, instead i use the block {% trans '....' %} in the tamplates and the function _('....') in the views – Warmblooded