Scrapy startproject command failed on import etree, Mac M1. The error message is: "symbol not found in flat namespace"

Question

I am trying to start a project using the Scrapy library, for a small webscraping project, but it fails on the import etree module. The exact error on the traceback is:

from .. import etree
ImportError: dlopen(/Users/myname/Desktop/scrapy_project/venv/lib/python3.10/site-packages/lxml/etree.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_exsltDateXpathCtxtRegister'

I have tried uninstalling reinstalling Scrapy and lxml libraries using pip, pip3, conda, and brew. Each time I am faced with the same error when I try the scrapy shell command or scrapy startproject command on the terminal. I have even downloaded xcode using the

xcode-select --install 

command, which did not seem to help either.

What exactly does the "symbol not found in flat namespace '_exsltDateXpathCtxtRegister'" error mean? Any idea how to work around the etree library or get scrapy to work? for reference I am working on a macbook pro m1 computer and running the project on Pycharm, using python 3.10. Thanks, any help would be greatly appreciated

Melee

Did you try the older Python version? Scrapy does not officially support 3.10. Also, do not use the system Python and or Homebrew. Download python 3.9 from python.org, create a virtual environment and then install scrapy. – Insular

Install libxml2 and libxslt before installing lxml. See this. – Insular

Answer 1

had the same issue, uninstall lxml: pip3 uninstall lxml

then

pip3 install lxml --no-cache-dir it will force redownload and build wheel

Briquet

Answer 2

I fixed this issue with uninstalling lxml and reinstalling it with conda:

pip uninstall lxml
conda -c install lxml

Watthour