Can't install xmlsec via pip

Question

I'm getting the following when running pip install xmlsec in macOS Big Sur 11.3.1:

Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/davidmasip/.pyenv/versions/3.9.9/bin/python3.9 /Users/davidmasip/.pyenv/versions/3.9.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/tmpm51b1yso
       cwd: /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d
  Complete output (65 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11.3-x86_64-cpython-39
  creating build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  copying src/xmlsec/py.typed -> build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  copying src/xmlsec/tree.pyi -> build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  copying src/xmlsec/__init__.pyi -> build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  copying src/xmlsec/constants.pyi -> build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  copying src/xmlsec/template.pyi -> build/lib.macosx-11.3-x86_64-cpython-39/xmlsec
  running build_ext
  building 'xmlsec' extension
  creating build/temp.macosx-11.3-x86_64-cpython-39
  creating build/temp.macosx-11.3-x86_64-cpython-39/private
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d
  creating build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_FTP=1 -DXMLSEC_NO_MD5=1 -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_CRYPTO_OPENSSL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.13 -I/usr/local/Cellar/libxmlsec1/1.3.0/include/xmlsec1 -I/usr/local/opt/[email protected]/include -I/usr/local/opt/[email protected]/include/openssl -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-build-env-gsnsoluq/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/Users/davidmasip/.pyenv/versions/3.9.9/include/python3.9 -c /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.c -o build/temp.macosx-11.3-x86_64-cpython-39/private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.c:319:5: error: use of undeclared identifier 'xmlSecSoap11Ns'
      PYXMLSEC_ADD_NS_CONSTANT(Soap11Ns, "SOAP11");
      ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.c:304:46: note: expanded from macro 'PYXMLSEC_ADD_NS_CONSTANT'
      tmp = PyUnicode_FromString((const char*)(JOIN(xmlSec, name))); \
                                               ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:19:19: note: expanded from macro 'JOIN'
  #define JOIN(X,Y) DO_JOIN1(X,Y)
                    ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:20:23: note: expanded from macro 'DO_JOIN1'
  #define DO_JOIN1(X,Y) DO_JOIN2(X,Y)
                        ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:21:23: note: expanded from macro 'DO_JOIN2'
  #define DO_JOIN2(X,Y) X##Y
                        ^
  <scratch space>:23:1: note: expanded from here
  xmlSecSoap11Ns
  ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.c:320:5: error: use of undeclared identifier 'xmlSecSoap12Ns'; did you mean 'xmlSecXPath2Ns'?
      PYXMLSEC_ADD_NS_CONSTANT(Soap12Ns, "SOAP12");
      ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/constants.c:304:46: note: expanded from macro 'PYXMLSEC_ADD_NS_CONSTANT'
      tmp = PyUnicode_FromString((const char*)(JOIN(xmlSec, name))); \
                                               ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:19:19: note: expanded from macro 'JOIN'
  #define JOIN(X,Y) DO_JOIN1(X,Y)
                    ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:20:23: note: expanded from macro 'DO_JOIN1'
  #define DO_JOIN1(X,Y) DO_JOIN2(X,Y)
                        ^
  /private/var/folders/ff/3y2196b13bq0nbm3_ms25nyh0000gp/T/pip-install-qm2a1dud/xmlsec_cd7a81ea26444cc4b8ae24acd3ec379d/src/common.h:21:23: note: expanded from macro 'DO_JOIN2'
  #define DO_JOIN2(X,Y) X##Y
                        ^
  <scratch space>:25:1: note: expanded from here
  xmlSecSoap12Ns
  ^
  /usr/local/Cellar/libxmlsec1/1.3.0/include/xmlsec1/xmlsec/strings.h:34:33: note: 'xmlSecXPath2Ns' declared here
  XMLSEC_EXPORT_VAR const xmlChar xmlSecXPath2Ns[];
                                  ^
  2 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for xmlsec
Failed to build xmlsec
ERROR: Could not build wheels for xmlsec which use PEP 517 and cannot be installed directly
WARNING: You are using pip version 21.2.4; however, version 23.0.1 is available.
You should consider upgrading via the '/Users/davidmasip/.pyenv/versions/3.9.9/bin/python3.9 -m pip install --upgrade pip' command.

I've also run before:

brew install libxml2 libxmlsec1 pkg-config xz

And I get:

Warning: libxml2 2.10.4 is already installed and up-to-date.
To reinstall 2.10.4, run:
  brew reinstall libxml2
Warning: libxmlsec1 1.3.0 is already installed and up-to-date.
To reinstall 1.3.0, run:
  brew reinstall libxmlsec1
Warning: pkg-config 0.29.2_3 is already installed and up-to-date.
To reinstall 0.29.2_3, run:
  brew reinstall pkg-config
Warning: xz 5.4.2 is already installed and up-to-date.
To reinstall 5.4.2, run:
  brew reinstall xz

how can I install xmlsec in macOS?

Cynthia

Answer 1

EDIT

This GitHub comment is the simplest fix.

https://github.com/xmlsec/python-xmlsec/issues/254#issuecomment-1612005910

---

When libxmlsec1 was bumped to v1.3 it broke pip install xmlsec.

https://github.com/xmlsec/python-xmlsec/issues/254

I am using a Macbook with Apple Silicon (M1 / M2) and struggled for a while to find this.

I managed to workaround it via the Homebrew formula hack in this comment by @dpritchett:

https://raw.githubusercontent.com/Homebrew/homebrew-core/7f35e6ede954326a10949891af2dba47bbe1fc17/Formula/libxmlsec1.rb

Specific workaround steps:

1. brew edit libxmlsec1. An editor opens up, full of the contents of the latest downloaded xmlsec formula from GitHub or wherever they come from
2. i paste in the contents of the last pre-1.3.0 brew formula and hit save
3. Install that local formula: brew install /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/libxmlsec1.rb
4. pip install xmlsec

~~EDIT~~

See comments below and comments on GitHub issue for additional info about the workaround that depend on your version of homebrew.

https://github.com/xmlsec/python-xmlsec/issues/254#issuecomment-1522646438

Athodyd

In my case the path to the formula was /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/libxmlsec1.rb – Kwashiorkor

Same here, path is /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/libxmlsec1.rb – Isiah

Additionally, I needed to uninstall libxmlsec1 before installing 1.2.37 again. brew uninstall libxmlsec1 You may want to add that as a step in your answer. – Isiah

I was struggling with this error for a while. Thank you! – Viperine

I had this issue in m2, I was able to resolve the issue by following this instruction. However in my case path to libxmlsec1.rb is /opt/homebrew/Cellar/libxmlsec1/1.3.2/.brew/libxmlsec1.rb – Playgoer

Had the same issue with Intel, however, with Python 3.10. And nothing here helps, but the original link to GitHub was posted at the beginning of this reply. Voted up. – Dink

Answer 2

There's another way to install an earlier version of libxmlsec1 that doesn't involve brew edit, written by D3X.

A "better" (cleaner) workaround is to create a local tap and extract the previous version, like so:

brew tap-new $USER/local
brew extract --version=1.2.37 libxmlsec1 $USER/local
brew uninstall libxmlsec1
brew install $USER/local/[email protected]

Once python-xmlsec is updated to support libxmlsec1 1.3+, you can uninstall the extracted version and remove the tap by doing:

brew uninstall $USER/local/[email protected]
brew untap $USER/local

Respirator

This worked on M2! Much appreciated! – Guesstimate

Answer 3

Answer by @Dag Høidahl almost worked for me. However the:

brew install $USER/local/[email protected]

command failed with 404 error:

Error: [email protected]: Failed to download resource "[email protected]"
Failure while executing; `/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.0.28-7-gcb61785\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 13.4.1\)\ curl/7.88.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --request GET https://www.aleksey.com/xmlsec/download/xmlsec1-1.2.37.tar.gz` exited with 22. Here's the output:
curl: (22) The requested URL returned error: 404
HTTP/1.1 404 Not Found
...

I had to manually patch the formula for it to work. I got the path to the formula from this command:

brew extract --version=1.2.37 libxmlsec1 $USER/local
==> Searching repository history
==> Writing formula for libxmlsec1 from revision 17a2c4d to:
/opt/homebrew/Library/Taps/user/homebrew-local/Formula/[email protected] # <- path to the formula

I then updated the file and replaced the following line:

url "https://www.aleksey.com/xmlsec/download/xmlsec1-1.2.37.tar.gz"

with:

url "https://www.aleksey.com/xmlsec/download/older-releases/xmlsec1-1.2.37.tar.gz"

So then rerunning brew install $USER/local/[email protected] worked fine.

Imperceptible

An alternative way to patch: run brew edit $USER/local/[email protected], update the URL to https://www.aleksey.com/xmlsec/download/older-releases/xmlsec1-1.2.37.tar.gz and then re-run brew install $USER/local/[email protected]. – Hardship

Answer 4

Since python-xmlsec hasn't been updated to support [email protected], creating a local tap of [email protected] and installing it is still the cleanest workaround for the issue. However, I ran into a few issues while using the solution mentioned by @Dag Høidahl.

1. Invalid tap name when running brew tap-new $USER/local My $USER has a . in it, making it an invalid tap name, so I had to manually set a name.
2. curl: (22) The requested URL returned error: 404 when running brew install $USER/local/[email protected]. The xmlsec tarball that brew downloads has moved into the older-releases directory with the release of version 1.2.38, so I needed to change the URL in the libxmlsec1.rb file.

This is an updated version that worked for me:

MYNAME=neruson
brew tap-new $MYNAME/local
brew extract --version=1.2.37 libxmlsec1 $MYNAME/local
sed -i '' 's^/xmlsec/download/^/xmlsec/download/older-releases/^g' /opt/homebrew/Library/Taps/$MYNAME/homebrew-local/Formula/[email protected]
brew uninstall libxmlsec1
brew install $MYNAME/local/[email protected]

Kickshaw