Mercurial commit messages and log, what encoding is supported, does Hg care at all?

Question

I tried doing a simple commit through my wrapper library for Mercurial, using the simple text of Unicode:æøåÆØÅ as my commit message. This is written to a text file and given to Mercurial with the appropriate parameter:

hg commit --logfile FILE

If I subsequently look at the repository with TortoiseHg, the characters are reproduced correctly. On the Console, they are mangled:

[C:\Temp] :hg log
changeset:   0:6a0911410128
tag:         tip
user:        Lasse V. Karlsen 
date:        Wed Dec 01 21:48:54 2010 +0100
summary:     Unicode:æøåÆØÅ

If I redirect the output of hg log to a file, and open it up, æøåÆØÅ is reproduced correctly.

So, my question is this:

• Can I ask hg to write the log to a file directly, or do I have to redirect standard output?
• Will this cause problems with python encoding for console, ie. some characters will make hg crash instead of just mangling the output?
• Is there a known supported encoding for the commit messages that I should adhere to?

Or is it just this simple:

• Mercurial doesn't care, it takes the contents of the file I give it, whatever the content, and stores that as the commit message. When producing the log, it will just dump it back to the console falling prey to whatever limitations the Python console output library has in this regard?

Frenchify

Answer 1

The following may not solve the issue but can help debug it.

Check out : https://www.mercurial-scm.org/wiki/EncodingStrategy

If I redirect the output of hg log to a file, and open it up, æøåÆØÅ is reproduced correctly.

So at least mercurial is storing the commit information correctly. It is only the output that is messed up.

Some work happening along these lines but not related to this

• http://bitbucket.org/mg/hg-unicode

[Edit: Missed the fact that you are on windows]

See the last paragraph on how to deal with character set compatibility problems: https://www.mercurial-scm.org/wiki/CharacterEncodingOnWindows

It says:

• set your console code page to match your system code page
• override Mercurial's encoding with an environment variable
  • Setting HGENCODING will override the detected system character set.
• override Mercurial's encoding with a command-line option
  • Using the global --encoding option will allow you to set your preferred encoding on each command.
• use GUI-based tools to interact with Mercurial
  • This also eliminates the problem, by eliminating that pesky console entirely.
• use Linux/UNIX and UTF-8
  • This makes Bill Gates cry.

Contrived

Not applicable: he's using Windows (C:\Temp). @Lasse: what console are you using? (Your prompt isn't the default Command Prompt prompt.) – Eurhythmics

Take Command, but does that matter? I'm spawning the program from .net and I assume (which is bad) that it uses cmd. Will check tomorrow, just reading on my iPad before bed now. – Frenchify

Also note that using the PROMPT command one can change the command prompt text, if that is what you picked up on. – Frenchify

@Chris Morgan: Thanks for pointing out what I missed to see. Yeah. The issue of encoding on windows looks a bit convoluted. – Contrived

Looks like that first URL indicate UTF8, will test some more tomorrow then. – Frenchify

@Lasse: the terminal you're using is the thing that matters most as it's what's actually drawing the characters. I know that the prompt can be changed; that's why I said it wasn't the default Command Prompt prompt rather than saying it wasn't Command Prompt (though that was most likely) – Eurhythmics

I don’t get that "Bill Gates" reference. UTF-8 is as much a de-facto standard on Windows as it is elsewhere. Notepad saves UTF-8. My editor saves UTF-8. Mercurial then goes in and messes it up on purpose, to "avoid making BG cry" and make me cry instead. – Autography

@romkyns the solution of "use Linux/UNIX and UTF-8" would be what makes bill gates cry, not UTF-8 itself. – Springwood

Answer 2

If using git bash for hg commands, you can set the Character set to ISO-8859-1 to fix that annoying chars in your hg log.

To do so, go to git bash options by clicking on the icon on the top left of the git bash * select options * select Text * At the bottom set the Character set to ISO-8859-1

I know it hurts to do so, but it works.

Fireside