M1 Mac - GDAL Wrong Architecture Error [Django]

Question

I'm trying to get a django project up and running, which depends on GDAL library. I'm working on a M1 based mac.

Following the instructions on official Django docs, I've installed the necessary packages via brew

$ brew install postgresql
$ brew install postgis
$ brew install gdal
$ brew install libgeoip

gdalinfo --version runs fine and shows the version as 3.3.1

gdal-config --libs returns this path: -L/opt/homebrew/Cellar/gdal/3.3.1_2/lib -lgdal

a symlink is also placed on the homebrew's lib directory, which is in my path env variable.

When I try to run django without specifying the path to gdal library, it complains that it cannot find the GDAL package (even though the library is reachable, as a symlink to it is available through path env variable).

When I try to specify the path to the GDAL library using GDAL_LIBRARY_PATH, I get this error:

OSError: dlopen(/opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib, 6): no suitable image found.  Did find:
    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib: mach-o, but wrong architecture
    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.29.dylib: mach-o, but wrong architecture

P.s. I've already seen this answer, but it didn't help.

Isn't that strange when I try to run gdalinfo it runs fine but when django tries to run it throws me this error? What am I doing wrong?

Maybellemayberry

Did you manage to find the answer to this as I've stumbled upon the same thing. – Melon

Also having the same issue, any update? – Piker

Sorry guys I've been away from stackoverflow for quite some time. We've ended up with running it with rosetta2. Today I'm doing another installation to another m1 machine, let's see if thigs have changed. – Maybellemayberry

Answer 1

GDAL and Python are likely compiled for different CPU architectures. On an M1 system the OS can run both native arm64 and emulated x86_64 binaries.

To check: run file /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib and file $(which python3), which should show the supported CPU architectures for both.

If the two don't match you'll have to reinstall one of them. Note that if you reinstall Python you also have to reinstall all Python packages with C extensions.

Arundinaceous

GDAL gives "Mach-O 64-bit dynamically linked shared library arm64" whereas Python gives "Bourne-Again shell script text executable, ASCII text". @Ronald -- how would you suggest reinstalling GDAL to make it compatible? – Sharpedged

Open your terminal using rosetta and do arch -x86_64 brew install gdal – Janey

GDAL gives /usr/local/homebrew/opt/gdal/lib/libgdal.dylib: Mach-O 64-bit dynamically linked shared library x86_64 and python gives two outputs, (for architecture x86_64): Mach-O 64-bit executable x86_64 and (for architecture arm64): Mach-O 64-bit executable arm64. So I am sure there is another work around in my case? – Conformal

Answer 2

I stumbled with the same issue, in my case it was solved by adding the GDAL_LIBRARY_PATH in the settings.py, but also GEOS_LIBRARY_PATH

GDAL_LIBRARY_PATH = '/opt/homebrew/Cellar/gdal/3.4.1_1/lib/libgdal.dylib'
GEOS_LIBRARY_PATH = '/opt/homebrew/Cellar/geos/3.10.2/lib/libgeos_c.1.16.0.dylib'

Baku

Yup, worked for me! Thanks @inaz. – Garald

Thanks @Inaz. Just a note to everyone: make sure you use libgeos_c not just libgeos. This tripped me up for a bit :) – Sharpedged

Answer 3

Try using the new arm version of python!

brew install --cask miniforge
conda init zsh
conda activate
conda install numpy scipy scikit-learn

Ribbing

Answer 4

Based on the insights from @ronald-oussoren, you have to make sure Python and Gdal were built from the same architecture. If you use a conda environment for Python for example, you can avoid the issue by installing gdal with conda (therefore you would have gdal available locally to your project) instead of homebrew:

conda install gdal

And you wouldn't need to manually setup the GDAL_LIBRARY_PATH or GEOS_LIBRARY_PATH settings.

Selfrestraint

Great information! – Piker

Answer 5

If you do not need this to run natively on the M1, consider using a Linux Virtual Machine.

My ultimate solution to this problem was to create an Ubuntu VM using Canonical's Multipass on my M1 Mac, then install postgresql, postgis, and all relevant dependencies including GDAL as one would for Linux.

https://multipass.run/

I used the following to install postgres and postgis:

sudo apt-get install libpq-dev #required for psycop2-binary installation
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

sudo apt-get -y install postgresql-13 #or other version number

sudo apt install postgis postgresql-13-postgis-3

sudo -i -u postgres
createuser yourusername
createdb postgis_db -O yourusername #create your db
psql -d postgis_db
CREATE EXTENSION postgis;

#make sure these are all installed:

sudo apt-get install binutils libproj-dev 
sudo apt-get install gdal-bin
sudo apt-get install libgeos++
sudo apt-get install proj-bin

I SSH into the ubuntu VM through VSCode and develop django per usual. There's a good article on Multipass setup here.

I've had no issues with this setup on M1.

UPDATE: As of 1/10/2022 there are some issues with MacOS's firewall on Monterey 12.1 not playing nice with Multipass. It's an open issue on GitHub and Canonical is working on it.

UPDATE #2: As of April 2022, it appears that multipass' issues with the MacOS firewall are resolved, following the latest OS update by Apple.

Piker

Answer 6

You don't need anything like arm 86_64 for installing gdal in mac M1. Need Anaconda for that. Set up is pretty simple compared to set set it up using arch -x86_64.

Download form here https://www.anaconda.com/download/

Then install anaconda on you M1 machine

1.Create new environment using python=3.8:

conda create --name python38 python=3.8

2.Active you environment:

conda activate python38

3.Install gdal using conda:

conda install -c conda-forge gdal

4.Install pygdal using pip:

pip install pygdal

Make sure pygdal and gdal version match with each other

Now use python38 as your interpreter where gdal is needed.

Partly