Updated the repo: [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/PyGraphviz/v1.5:
Using official Graphviz 2.42.2 sources
.whls (win_amd64, win32) for currently supported Python versions
Newer versions might be added (check one level up)
For Python 2.7, they are already built: [UCI.LFD]: Unofficial Windows Binaries for Python Extension Packages - PyGraphviz, an interface to the Graphviz graph layout and visualization package..
Notes:
In some (I guess, most of the) cases, a Graphviz installation will be required on the system where PyGraphviz runs on, because PyGraphviz uses some of Graphviz's tools (executables). They can be downloaded, or built (they don't have to match PyGraphviz architecture (032bit (PC032), 064bit (PC064)), as they are invoked).
Update: I also added Graphviz 2.42.2 build (PC032 - as it works on both PC064 and PC032 Win) in the above repository
- Check it for newer software versions
Also, a bug (present in previous versions) was fixed. Check [SO]: pygraphviz 1.5 default edge no arrow? (@CristiFati's answer) for more details
Anyone who wants to know more details about the build process, read on!
1. Intro
Almost 2 years later, and the problem (well, not exactly as in the question) still persists.
I want to start by emphasizing the difference between the 2 packages:
In Anaconda environment, [SO]: Installing PyGraphviz on Windows 10 64-bit, Python 3.6 (@TomHanks's answer) works perfectly.
PyGraphwiz only has available for download an archive (.zip, in this case) file, meaning that it contains (C / C++) sources.
A couple of words about packages (.whls) whose names contain things like cp34-none-win_amd64 (check [SO]: What does version name 'cp27' or 'cp35' mean in Python? (@WayneWerner's answer) for details):
Now, many packages have prebuilt binaries for most common Python versions running on various OSes (e.g. [PyPI]: mysql-connector-python - Download files), but just as many don't, and those only contain sources. Unfortunately, PyGraphviz is in the 2nd category. For the latter ones, pip install will:
As a side note: pip -v ... enables verbose mode for the current command, which comes in extremely handy when experiencing install errors.
Back to our problem: Python 3.6 needs VStudio 2015 ([Python.Wiki]: WindowsCompilers).
This is a very vast topic, I covered some parts in:
You should check them before proceeding and also keep them open, as you will definitely need them in the next steps.
I have VStudio 2015 Community (among many other versions) installed, you should install it too, it's free ([MS.VStudio]: Still want an older version?).
PyGraphviz depends on [Graphviz]: Graph Visualization Software. So, at build time it will need (parts of) Graphviz (which also has other dependencies of its own) to be already built. Unfortunately, I couldn't find prebuilt binaries (there is [Graphviz]: Windows Packages - graphviz-2.38.zip, but that's not helping), so it will have to be built manually.
Before going further:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" -c "import pygraphviz"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pygraphviz'
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> dir /b
other
src
This is my top dir, any sources are downloaded in the src dir, binaries will be placed in the bin dir.
2. Build Graphviz
Before starting, I want to mention that I heavily rely on Cygwin (you don't have to, there are also other variants (MSYS2)), and some of my tools are installed there, so I'll be alternating between Cygwin and Cmd terminals (which might be confusing).
[Graphviz]: Graphviz Build Instructions for Windows states:
For building on Windows:
(Graphviz versions ≥ 2.41)
First, in the root of the repository, perform git submodule update --init. This will download all submodules, which are mostly the dependencies for the Windows build. Next, add the windows\dependencies\graphviz-build-utilities directory to your PATH (and restart Visual Studio or the prompt with which you execute msbuild after that). This folder contains the tools Bison, Flex and SED (and future additions) with versions that are tested. If all went right, the dependencies are now set up and you can build Graphviz.
First, we need to download everything:
[cfati@cfati-5510-0:/cygdrive/e/Work/Dev/StackOverflow/q045093811/src/graphviz]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> git clone https://gitlab.com/graphviz/graphviz.git .
Cloning into '.'...
remote: Enumerating objects: 71728, done.
remote: Counting objects: 100% (71728/71728), done.
remote: Compressing objects: 100% (19331/19331), done.
remote: Total 71728 (delta 52200), reused 71681 (delta 52157)
Receiving objects: 100% (71728/71728), 163.79 MiB | 480.00 KiB/s, done.
Resolving deltas: 100% (52200/52200), done.
Checking out files: 100% (3870/3870), done.
[064bit prompt]>
[064bit prompt]> git submodule update --init
Submodule 'dependencies/criterion' (https://github.com/Snaipe/Criterion.git) registered for path 'dependencies/criterion'
Submodule 'windows/dependencies/graphviz-build-utilities' (https://github.com/ErwinJanssen/graphviz-build-utilities.git) registered for path 'windows/dependencies/graphviz-build-utilities'
Submodule 'windows/dependencies/libraries' (https://github.com/ErwinJanssen/graphviz-windows-dependencies.git) registered for path 'windows/dependencies/libraries'
Cloning into '/cygdrive/e/Work/Dev/StackOverflow/q045093811/src/graphviz/dependencies/criterion'...
Cloning into '/cygdrive/e/Work/Dev/StackOverflow/q045093811/src/graphviz/windows/dependencies/graphviz-build-utilities'...
Cloning into '/cygdrive/e/Work/Dev/StackOverflow/q045093811/src/graphviz/windows/dependencies/libraries'...
Submodule path 'dependencies/criterion': checked out '301d143ea42c024f22b673b69c72a4cb3c8d151f'
Submodule path 'windows/dependencies/graphviz-build-utilities': checked out '050fff84ce195e0740878748760fd801eeb07b23'
Submodule path 'windows/dependencies/libraries': checked out '141d3a21be904fa8dc2ae3ed01d36684db07a35d'
[064bit prompt]>
[064bit prompt]> git show head
commit 89292b5945933b1501293c04894ed9cf886241be (HEAD -> master, origin/master, origin/HEAD)
Merge: 429d43615 97811bd35
Author: Stephen C North <[email protected]>
Date: Mon Feb 4 08:09:40 2019 -0500
Merge branch 'wasbridge/graphviz-master' into HEAD
[064bit prompt]> git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
You'll end up with a dir that contains ~320 MiB of stuff. The dir contains a graphviz.sln file, which is a VStudio (2015) solution file which contains 63 projects.
Looking at the Anaconda or Python 2.7 PyGraphviz (built) package, it only depends on cgraph.dll, which in turn depends on cdt.dll, so only the 2 projects are relevant to us. Note that these 2 projects might not need all the Git submodules (so the dir might be trimmed down), but I didn't investigate further.
Unfortunately, the projects are only configured for 032bit (Win32 platform (pc032)). The 064bit one must be manually added (I did it from VStudio IDE - and also described the process in one of my answers that I referenced). After saving the projects, they will be shown as modified by Git:
[064bit prompt]> git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
(commit or discard the untracked or modified content in submodules)
modified: lib/cdt/cdt.vcxproj
modified: lib/cgraph/cgraph.vcxproj
modified: windows/dependencies/graphviz-build-utilities (modified content)
no changes added to commit (use "git add" and/or "git commit -a")
The 3rd item is because I needed to reset some security permissions on 2 executables (used when building cgraph):
which were not set properly (most likely, because of Cygwin).
You can build the 2 projects from IDE, but I chose command line ([MS.Docs]: MSBuild command-line reference) since I find it more flexible:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64
[prompt]> set PATH=%PATH%;%CD%\src\graphviz\windows\dependencies\graphviz-build-utilities
[prompt]> msbuild src\graphviz\lib\cdt\cdt.vcxproj /t:Rebuild /p:Platform=x64;Configuration=Release;SolutionDir=%CD%\src\graphviz\;OutDir=%CD%\bin\Win\dynamic\064\UCRTv140\md\Release\graphviz\ >build_cdt_064.txt 2>&1
[prompt]> echo %errorlevel%
0
[prompt]> dir /b
bin
build_cdt.txt
other
src
[prompt]> msbuild src\graphviz\lib\cgraph\cgraph.vcxproj /t:Rebuild /p:Platform=x64;Configuration=Release;SolutionDir=%CD%\src\graphviz\;OutDir=%CD%\bin\Win\dynamic\064\UCRTv140\md\Release\graphviz\ >build_cgraph_064.txt 2>&1
[prompt]> echo %errorlevel%
0
[prompt]> dir /b "bin\Win\dynamic\064\UCRTv140\md\Release\graphviz"
cdt.dll
cdt.dll.lastcodeanalysissucceeded
cdt.exp
cdt.lib
cgraph.dll
cgraph.dll.lastcodeanalysissucceeded
cgraph.exp
cgraph.lib
So, we have everything needed (2 .lib and 2 .dll files) in order to go on.
3. Build PyGraphviz
PyGraphviz sources are (downloaded from [GitHub]: pygraphviz/pygraphviz - (pygraphviz-1.5) pygraphviz-pygraphviz-1.5.zip and) unpacked in src/pygraphviz/pygraphviz-pygraphviz-1.5.
One more adjustment is needed to Graphviz (probably it's done as part of another project - an install step): preparing the header files:
[prompt]> mkdir include\graphviz
[prompt]> copy src\graphviz\lib\cdt\cdt.h include\graphviz
1 file(s) copied.
[prompt]> copy src\graphviz\lib\cgraph\cgraph.h include\graphviz
1 file(s) copied.
Unfortunately, PyGraphviz does not build OOTB, because of [GitHub]: pygraphviz/pygraphviz - Python 3 support. To fix that, [GitHub]: eendebakpt/pygraphviz - Workaround for PyIOBase_Type for Python2 on win must be applied. I adapted it to work with the current sources (as it doesn't work OOTB, as well :X( ) for graphviz_wrap.cpp only:
pygraphviz-1.5-all-pyiobase_b85d12ac22d39063f7dbcc396e825c563431e352.patch:
--- pygraphviz/graphviz_wrap.c.orig 2018-09-10 16:07:12.000000000 +0300
+++ pygraphviz/graphviz_wrap.c 2019-02-26 18:05:20.281741400 +0200
@@ -2988,7 +2988,18 @@
#if PY_VERSION_HEX >= 0x03000000
-extern PyTypeObject PyIOBase_Type;
+static PyObject *PyIOBase_TypeObj;
+
+static int init_file_emulator(void)
+{
+ PyObject *io = PyImport_ImportModule("_io");
+ if (io == NULL)
+ return -1;
+ PyIOBase_TypeObj = PyObject_GetAttrString(io, "_IOBase");
+ if (PyIOBase_TypeObj == NULL)
+ return -1;
+ return 0;
+}
#endif
@@ -3449,7 +3460,7 @@
{
#if PY_VERSION_HEX >= 0x03000000 || defined(PYPY_VERSION)
#if !defined(PYPY_VERSION)
- if (!PyObject_IsInstance(obj0, (PyObject *)&PyIOBase_Type)) {
+ if (!PyObject_IsInstance(obj0, PyIOBase_TypeObj)) {
PyErr_SetString(PyExc_TypeError, "not a file handle");
return NULL;
}
@@ -3523,7 +3534,7 @@
{
#if PY_VERSION_HEX >= 0x03000000 || defined(PYPY_VERSION)
#if !defined(PYPY_VERSION)
- if (!PyObject_IsInstance(obj1, (PyObject *)&PyIOBase_Type)) {
+ if (!PyObject_IsInstance(obj1, PyIOBase_TypeObj)) {
PyErr_SetString(PyExc_TypeError, "not a file handle");
return NULL;
}
@@ -6051,6 +6062,12 @@
SWIG_InstallConstants(d,swig_const_table);
+#if PY_VERSION_HEX >= 0x03000000
+ if (init_file_emulator() < 0) {
+ return NULL;
+ }
+#endif
+
PyDict_SetItemString(md,(char*)"cvar", SWIG_globals());
SWIG_addvarlink(SWIG_globals(),(char*)"Agdirected",Swig_var_Agdirected_get, Swig_var_Agdirected_set);
SWIG_addvarlink(SWIG_globals(),(char*)"Agstrictdirected",Swig_var_Agstrictdirected_get, Swig_var_Agstrictdirected_set);
That is a diff (patch). See [SO]: Run/Debug a Django application's UnitTests from the mouse right click context menu in PyCharm Community Edition? (@CristiFati's answer) (Patching UTRunner section) for how to apply patches on Win (basically, every line that starts with one "+" sign goes in, and every line that starts with one "-" sign goes out).
[prompt]> :: Restore the original prompt as cwd is important
[prompt]> exit
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> set _TOP_DIR=%CD%
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> pushd src\pygraphviz\pygraphviz-pygraphviz-1.5
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811\src\pygraphviz\pygraphviz-pygraphviz-1.5]> pushd pygraphviz && "c:\Install\x64\Cygwin\Cygwin\AllVers\bin\patch.exe" -p 1 -buNi ..\pygraphviz-1.5-all-pyiobase_b85d12ac22d39063f7dbcc396e825c563431e352.patch && popd
patching file graphviz_wrap.c
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811\src\pygraphviz\pygraphviz-pygraphviz-1.5]> echo %errorlevel%
0
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811\src\pygraphviz\pygraphviz-pygraphviz-1.5]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" setup.py install --include-path=%_TOP_DIR%\include --library-path=%_TOP_DIR%\bin\Win\dynamic\064\UCRTv140\md\Release\graphviz >%_TOP_DIR%\install_pygraphviz_064.txt 2>&1
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811\src\pygraphviz\pygraphviz-pygraphviz-1.5]> echo %errorlevel%
0
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811\src\pygraphviz\pygraphviz-pygraphviz-1.5]> popd
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> set PATH=%PATH%;%CD%\bin\Win\dynamic\064\UCRTv140\md\Release\graphviz
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q045093811]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" -c "import pygraphviz;print(dir(pygraphviz), \"\n\", pygraphviz.graphviz._graphviz)"
['AGraph', 'Attribute', 'DotError', 'Edge', 'ItemAttribute', 'Node', '__all__', '__author__', '__builtins__', '__cached__', '__date__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__revision__', '__spec__', '__version__', 'absolute_import', 'agraph', 'division', 'graphviz', 'print_function', 'release', 'test', 'tests', 'version']
<module '_graphviz' (e:\Work\Dev\VEnvs\py_064_03.06.08_test0\lib\site-packages\pygraphviz\_graphviz.cp36-win_amd64.pyd)>
As seen, the module was successfully imported.
As a remark, the 2 .dll dependencies (from previous section) must be available when the module is imported, so their dir is added to %PATH%.
Of course this is only a (lame) workaround (gainarie), this shouldn't happen every time one has to work with the package.
I don't know (yet) how to instruct setup.py to also copy them in the package build / install dir, so as an alternative (also workaround) one has to manually copy them in PyGraphviz install dir (next to _graphviz.cp36-win_amd64.pyd, which is (in my case): "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\lib\site-packages\pygraphviz").
4. Shortcut
Since the whole process is complex and requires lots of manual interventions and hacks, I've managed to build (with minor setup.py modifications) the (.whl) package.
I am not aware of a simple way to make it publicly available, so (although maybe it's not the best practice,) I uploaded it at [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/PyGraphviz/v1.5/pygraphviz-1.5-cp36-cp36m-win_amd64.whl.
Notes:
The following is regarding PyGraphviz, but it can be generalized and applied to any (.whl) package
Also, the GitHub repository will be used as an example, but (again) it can apply to (.whls in) any repository, including [PyPI]: The Python Package Index (I'm not using the PyGraphviz one as it has no binaries, but for example [PyPI]: numpy 1.17.0 - Download files)
You can download it locally (e.g. in C:\Path\to\downloaded), then install it like (this is one way, check [SO]: How to install a package for a specific Python version on Windows 10? (@CristiFati's answer) for alternatives):
"C:\Path\to\Python-3.6-amd64\python.exe" -m pip install "C:\Path\to\downloaded\pygraphviz-1.5-cp36-cp36m-win_amd64.whl"
The above .whl is located next to several others. Those are for different Python versions (and that happens when the .whl contains .dlls (.pyds) or .sos (on Nix) that link to a specific Python library). To match your Python version search the .whl name for cp${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR}. In the current situation, it's cp36 corresponding to Python 3.6, but the same principle applies for other Python versions (cp37, cp38, cp39, cp310, cp311, ...). Check [SO]: What does version name 'cp27' or 'cp35' mean in Python? for more details.
Same thing for the architecture (win_amd64 / win32 (x86_64 / i686 on Nix)).
Note: It works for Anaconda environments as well!
Pip install pygraphvizdoesn't work for me on both 2.7 and 3.6. I was able to install 2.7 by downloading the cp27 whl file. So i'm looking for the cp34 file for python 3. – Arkansas