How do I correctly install dulwich to get hg-git working on Windows?

Question

I'm trying to use the hg-git Mercurial extension on Windows (Windows 7 64-bit, to be specific). I have Mercurial and Git installed. I have Python 2.5 (32-bit) installed.

I followed the instructions on http://hg-git.github.com/ to install the extension. The initial easy_install failed because it was unable to compile dulwich without Visual Studio 2003.

I installed dulwich manually by:

• git clone git://git.samba.org/jelmer/dulwich.git
• cd dulwich
• c:\Python25\python setup.py --pure install

Now when I run easy_install hg-git, it succeeds (since the dulwich dependency is satisfied).

In my C:\Users\username\Mercurial.ini, I have:

[extensions]
hgext.bookmarks =
hggit =

When I type 'hg' at a command prompt, I see: "*** failed to import extension hggit: No module named hggit"

Looking under my c:\Python25 folder, the only reference to hggit I see is Lib\site-packages\hg_git-0.2.1-py2.5.egg. Is this supposed to be extracted somewhere, or should it work as-is?

Since that failed, I attempted the "more involved" instructions from the hg-git page that suggested cloning git://github.com/schacon/hg-git.git and referencing the path in my Mercurial configuration. I cloned the repo, and changed my extensions file to look like:

[extensions]
hgext.bookmarks =
hggit = c:\code\hg-git\hggit

Now when I run hg, I see: *** failed to import extension hggit from c:\code\hg-git\hggit: No module named dulwich.errors.

Ok, so that tells me that it is finding hggit now, because I can see in hg-git\hggit\git_handler.py that it calls

from dulwich.errors import HangupException

That makes me think dulwich is not installed correctly, or not in the path.

Update:

From Python command line:

import dulwich

yields Import Error: No module named dulwich

However, under C:\Python25\Lib\site-packages, I do have a dulwich-0.5.0-py2.5.egg folder which appears to be populated. This was created by the steps mentioned above. Is there an additional step I need to take to make it part of the Python "path"?

From Python command line (as suggested in one of the answers):

import pkg_resources
pkg_resources.require('dulwich')

yields [dulwich 0.5.0 (c:\python25\lib\site-packages\dulwich-0.5.0-py2.5.egg)]

So what does that tell me? Importing dulwich fails, but apparently pkg_resources can find it. What can I do with that information?

Quake

In an interactive Python session, run: {{{ >>> import dulwich >>> print dulwich.__version__ }}} What does that show? – Garofalo

Added results to the Update section above – Quake

I am assuming that you are asking about Microsoft's Window (not just any windowing system). If anyone coming to this question from Debian (or similar) then you can do sudo apt install python-dulwich – Vaporimeter

Answer 1

That makes me think dulwich is not installed correctly, or not in the path.

You're absolutely right. Mercurial binary distributions for Windows are 'frozen' - they use the Python code and interpreter bundled with them and therefore independent of packages installed in system PYTHONPATH. When you specify path to hggit extension in Mercurial.ini, hg tries to import it using direct path, but dulwich library is not imported explicitly by hg and doesn't bundled with its library, so the import fails.

It is possible to add both Dulwich and HgGit into library.zip that is installed along with hg.exe, but for me the best way is to install everything from source including Mercurial and execute commands using .bat files installed into \Python\Scripts. In this case you will need to:

1. Install Mercurial from source. This builds "pure" version, because Windows users usually don't have Visual Studio or alternative compiler for compiling C speedups.

2. Install Dulwich - I'd use latest trunk snapshot for both Git and Dulwich.

   python setup.py --pure install

3. Install latest HgGit snapshot

   python setup.py install

4. Edit Mercurial.ini to enable hggit =

5. Launch Mercurial using your \Python\Scripts\hg.bat

Foolproof

What's wrong with adding Dulwich and HgGit into library.zip? It looks like nicer solution to me. – Effeminize

@Piotr: Nothing wrong except that library.zip is not always an ordinary archive. So you will need a script like this - techtonik.rainforce.org/2010/01/… – Foolproof

Or if using Debian sudo apt install python-dulwich. (I found this is a search, but I don't use MS) – Vaporimeter

Answer 2

I found a simpler solution at http://candidcode.com/2010/01/12/a-guide-to-converting-from-mercurial-hg-to-git-on-a-windows-client/

And then I found a yet simpler solution myself:

To use the hg-git Mercurial extension on Windows:

1. install the official Mercurial binaries
2. put dulwich folder from dulwich sources and hggit folder from hg-git sources to the root of library.zip in Mercurial installation folder
3. add the following to %USERPROFILE%\Mercurial.ini:

---

[extensions]
hgext.bookmarks=
hggit=

To have SSH support you need plink.exe from PuTTY family. After that you should add the following to Mercurial.ini:

[ui]
username = John Doe <[email protected]>
ssh=d:/home/lib/dll/plink.exe -i "d:/home2/ssh-private-key.ppk"

When connecting to a SSH server for the first time, you should start putty.exe and try to connect using it. It will add the server key fingerprint to the registry. Otherwise plink will ask you to accept the fingerprint but it doesn't accept any input.

You can use puttygen.exe to generate private keys. Either use keys without a passphrase or use Pageant.exe ssh authentication agent.

Kra

I'm clueless with Python, so all the other methods I looked at I either totally failed at, or didn't attempt. This one got it sorted, no problem. Thanks! – Diviner

In the wake of bitbucket dropping mercurial support recently, this is more relevant than ever and worked for me! – Vogler

I'm glad it still works after 8 years of potential code rot. – Kra

Answer 3

If you can install TortoiseHg, it includes dulwich and other requirements.

Ils

@David did you install other Mercurial executable? Please make sure your "hg" command points TortoiseHg's Mercurial executable. or try TortoiseHg 1.0.4. – Ils

This worked for me, following the instructions here – Insightful

Answer 4

Try following configuration (change to your path), which works for me:

[extensions]
; hg-git extention
hgext.bookmarks =
hggit = C:\Python26\Lib\site-packages\hg_git-0.2.1-py2.6.egg\hggit

In my case when I have empty value for hggit =, I get the same error as you do in this case. But I can import dulwich without problem in python shell, so you should check your easy-install.pth (as pointed out by David) if it contains dulwich-0.5.0-py2.5.egg. I did install pure version of dulwich as well.

Outfox

Do I need to extract the egg for that to work? In my site-packages directory, I have that .egg file, but I do not have a directory named hg_git-0.2.1-py2.6.egg. Are you referencing hggit within the unexpanded egg file? – Quake

I do not have the egg file, and if I remember correctly the .egg file did not work properly on my machine. Then I installed using -Z parameter of easy_install in order to extract it during the installation. The same applies for dulwich. Hope it helps you. – Outfox

This worked for me with Tortoise 1.0.3 Hg 1.5.3 installed. Make sure you use the -Z when you easy_install hg-git, and correct the hggit path above to whatever version it ends up downloading – Schramm

Answer 5

I got this error as well even after downloading the latest Tortoisehg and making sure the hggit plugin was installed as well as my .ini & hgrc files had the right entry to enable hggit.

Turns out my problem was that I had both mercurial and tortoisehg in my path. So when I ran any hg commands, it was using the hg.exe in mercurial folder as opposed to the hg.exe in the torsoisehg directory.

This makes sense but my mercurial installation did not have the plug ins. My fix was to remove mercurial from my path so hg commands go through the tortoisehg directory since it has hg completely bundled. Note however, the recommended option might be to upgrade mercurual to a version that has the plugins that one needs but this is what worked for me. I tried replacing the library.zip in mercurial with the one in tortoisehg and this worked but it led to other errors as one would imagine.

@techtonik's answer led me down this road for which I am grateful.

Recap: verify which hg exe is running your hg commands by checking your path because that hg.exe does not find the plugins for whatever reason.

Bagasse

Answer 6

Had this problem today when installing the latest TortoiseHg.

Get the latest python 2.7 (I used 2.7.16) setup on your system, you probably have this already. Get a command window and go to the c:\Python27 folder To run pip use Scripts\pip or easy_install use Scripts\easy_install Try pip -V to make sure you get the 2.7 version and not some 3.X version

Wrong:

c:\Python27>pip -V
pip 20.2.4 from c:\python38\lib\site-packages\pip (python 3.8)

Right:

c:\Python27>Scripts\pip -V
pip 20.2.4 from c:\python27\lib\site-packages\pip (python 2.7)

If dulwich or hg-git are installed already

Scripts\pip uninstall dulwich
Scripts\pip uninstall hg-git

Install hg-git

Scripts\easy_install install hg-git

You should now have two folders

C:\Python27\Lib\site-packages\dulwich-0.19.16-py2.7-win-amd64.egg\dulwich
C:\Python27\Lib\site-packages\hg_git-0.9.0-py2.7.egg\hggit

It will only work if Dulwich is version 0.19.16 (less than 0.20.x)

Copy these folders (dulwich and hggit) into the zip-file C:\Program Files\TortoiseHg\lib\library.zip

folder in zip file

Huntress

In my case, pip install hg-git (not hggit) was sufficient to get hg-git and all dependencies (dulwich, certifi and urllib3) installed in the site_packages folder. Copying them into library.zip indeed works.Thank you! – Pelorus

Answer 7

Until you get import dulwich to work, hggit won't work. Check that the dulwich egg file is in your easy-install.pth file under site-packages.

For further debugging you can try ask pkg_resources about it:

import pkg_resources
pkg_resources.require("dulwich")

Cornstalk

This line is in my easy-install.pth file: /c/Python25/lib/site-packages/dulwich-0.5.0-py2.5.egg – Quake

I ran the pkg_resources command and put the results in the question above. Now what? – Quake

I can reproduce your exact result by easy_installing dulwich and removing it from easy-install.pth Did you manually edit the path in your easy-install - the /c/Python25 looks wrong - I'd replace that with a straight dulwich-0.5.0-py2.5.egg Or even better, remove the line and re-run easy_install dulwich – Cornstalk

Answer 8

I ran into this problem too with dulwich.errors. Instead of installing everything from scratch. I just copied dulwich from my default site-packages to the mercurial site-packages. worked with no problems.

Instauration

This will work, but it means you won't be compiling the C extensions. Dulwich (and thus hg-git) will use the pure python implementation and thus be (significantly) slower. – Academe

Answer 9

Based on techtonik's explanation of the reason for the failing import of dulwich.errors, I found a solution which appears simpler to me than the already proposed ones:

On the Mercurial download page there is offered

Mercurial <x.y.z> for Python 2.7 on Windows x86 (source install)

and

Mercurial <x.y.z> for Python 2.7 on Windows x64 (source install)

as .EXE files which install Mercurial as a Python module into an existing Python 2.7 (x86 or x64) installation.

If hg-git and dulwich have been installed to this Python installation as well, the import errors should vanish.

If you are not sure what to do now, I hope this step-by-step explanation helps:

Prerequisites

• Python 2.7 is installed
• <python 2.7 install dir> and <python 2.7 install dir>\Scripts are in the PATH environment variable
• hg-git (and dulwich) have been installed into this Python installation via the instructions from here

Steps

1. Deinstall any existing Mercurial installation
2. Download and install one of the above-quoted EXE files, depending on whether your Python 2.7 installation is 32bit or 64bit (If you get a "Python 2.7 cannot be found in registry" error, you probably need the other one.)

Now hg clone <some git repo> should work on the console.

Allowable

Answer 10

After a long research I finally managed to setup hggit properly.
What I have done:

1. I have installed the newest version of TortoiseHg (in my case Version 5.9.1)
   This official version (automatically) contains Python-2.7.18 (According to Help/About)
2. Download dulwich: cd C:\dulwich, git clone https://github.com/dulwich/dulwich.git .
3. TortoiseHg uses Python 2.7, dulwich doesn't support this (old) Python version anymore. We have to go back to an older version of dulwich (<0.20)
   git reset --hard c6993451a745d313f61e5d080b9f9d8611a8d7f4
4. The subfolder C:\dulwich\dulwich has to be copied inside the zip file
   C:\Program Files\TortoiseHg\lib\library.zip so that there is a path
   C:\Program Files\TortoiseHg\lib\library.zip\dulwich
   This can be done by the following commands:
   a) Extract the currenty content of TortoiseHg's library
   md "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
tar -xf "C:\Program Files\TortoiseHg\lib\library.zip" -C "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
   b) Copy dulwich files into directory
   xcopy C:\dulwich\dulwich "C:\Program Files\TortoiseHg\lib\ExtractedLibrary\dulwich\"
   c) Save old library.zip and create new library.zip
   ren "C:\Program Files\TortoiseHg\lib\library.zip" library_old.zip
cd "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
tar -acf "C:\Program Files\TortoiseHg\lib\library.zip" *.*
   d) Cleanup
   cd ..
rd /q /s "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
rd /q /s C:\dulwich
5. Download hg-git: cd C:\hg-git, hg clone https://foss.heptapod.net/mercurial/hg-git .
6. The current version of hg-git also does not support Python 2.7 anymore. We have to go back to the last working version 0.9.0 (=revision 1320 in TortoiseHg):
   hg update 6d7d6f174df8
7. Create a new empty repository which will be "connected" to a remote git repository
   cd C:\MyLocalHgRepoWhichIsConnectedToGit, hg init
8. Create a textfile C:\MyLocalHgRepoWhichIsConnectedToGit\.hg\hgrc with the following content:

[paths]
default = C:\ConnectedGitRepoForPushAndPull

[extensions]
hgext.bookmarks = 
hggit = C:\hg-git\hggit

Remark: Alternatively you can also copy the folder C:\hg-git\hggit inside library.zip and leave the path in the hgrc file empty. Indeed this is my favourite way. Do it the same way as I have explained under step 4 with dulwich.

That's it. Now the Mercurial Repo is linked to the Git Repo
and you can execute hg pull or hg push inside the Hg Repo.

---

---

If you don't want to read all my explanations you can simply use this batch file and execute it:

hggit.bat

@echo off
rem Installation von dulwich+hggit für TortoiseHg
rem Michael Hutter, 22.09.2021
rem https://mcmap.net/q/325114/-how-do-i-correctly-install-dulwich-to-get-hg-git-working-on-windows

if "%1"=="install" goto install
if "%1"=="clonegitrepo" goto clonegitrepo
echo Syntax:
echo %0 install (installiert dulwich und hggit in TortoiseHg)
echo %0 clonegitrepo C:\HgRepoToCreate C:\GitRepoToClone (Klonen eines Git-Repos in ein Hg-Repo)
pause
goto ende


:install
set LibraryPath=C:\Program Files\TortoiseHg\lib
set TempPathDulwich=C:\Temp\dulwich
set TempPathHggit=C:\Temp\hg-git

if not exist "%LibraryPath%\library.zip" (
echo Die Datei %LibraryPath%\library.zip existiert nicht!
pause
goto Ende
)

rem Installation von dulwich
md %TempPathDulwich%
cd %TempPathDulwich%
git clone https://github.com/dulwich/dulwich.git .
git reset --hard c6993451a745d313f61e5d080b9f9d8611a8d7f4
pause

rem Installation von hg-git
md %TempPathHggit%
cd %TempPathHggit%
hg clone https://foss.heptapod.net/mercurial/hg-git .
hg update 6d7d6f174df8
pause

rem dulwich und hggit in library.zip aufnehmen
md "%LibraryPath%\ExtractedLibrary"
tar -xf "%LibraryPath%\library.zip" -C "%LibraryPath%\ExtractedLibrary"
xcopy %TempPathDulwich%\dulwich "%LibraryPath%\ExtractedLibrary\dulwich\"
xcopy %TempPathHggit%\hggit "%LibraryPath%\ExtractedLibrary\hggit\"
ren "%LibraryPath%\library.zip" library_old.zip
if exist "%LibraryPath%\library.zip" del "%LibraryPath%\library.zip"
cd "%LibraryPath%\ExtractedLibrary"
tar -acf "%LibraryPath%\library.zip" *.*

rem Aufräumen
cd ..
rd /q /s "%LibraryPath%\ExtractedLibrary"
rd /q /s %TempPathDulwich%
rd /q /s %TempPathHggit%
pause
goto ende


:clonegitrepo
rem Klonen eines Git-Repos in ein lokales Hg-Repo
echo Erstelle neues HgRepo %2
md %2
cd %2
if not exist %2 (
echo "Fehler: %2 existiert nicht!"
pause
goto ende
)
hg init
echo [paths] > .hg\hgrc
echo default = %3 >> .hg\hgrc
echo. >> .hg\hgrc
echo [extensions] >> .hg\hgrc
echo hgext.bookmarks = >> .hg\hgrc
echo hggit = >> .hg\hgrc
hg pull
hg update
pause

:ende

Polydipsia

Answer 11

sudo apt-get install python-dev # Dam you dependency!

sudo easy_install dulwich

Success!

Malebranche