arch x86_64 and arm64e is available but python3 is saying incompatible architecture on Mac M1

Question

I am trying to run this reading-text-in-the-wild on Mac M1. When I attempt to run this code

 python3 make_keras_charnet_model.py

I get the error

Using Theano backend.
Traceback (most recent call last):
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 305, in <module>
    from cutils_ext.cutils_ext import *  # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 316, in <module>
    from cutils_ext.cutils_ext import *  # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "make_keras_charnet_model.py", line 10, in <module>
    from keras.models import Sequential, model_from_json
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/models.py", line 15, in <module>
    from . import backend as K
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/__init__.py", line 47, in <module>
    from .theano_backend import *
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/theano_backend.py", line 1, in <module>
    import theano
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/__init__.py", line 76, in <module>
    from theano.scan_module import scan, map, reduce, foldl, foldr, clone
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/__init__.py", line 40, in <module>
    from theano.scan_module import scan_opt
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/scan_opt.py", line 59, in <module>
    from theano import tensor, scalar
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/__init__.py", line 7, in <module>
    from theano.tensor.subtensor import *
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/subtensor.py", line 27, in <module>
    import theano.gof.cutils  # needed to import cutils_ext
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 319, in <module>
    compile_cutils()
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 283, in compile_cutils
    cmodule.GCC_compiler.compile_str('cutils_ext', code, location=loc,
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 2212, in compile_str
    return dlimport(lib_filename)
  File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 299, in dlimport
    rval = __import__(module_name, {}, {}, [module_name])
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)

I have duplicated my terminal to have the duplicate open with Rosetta and still I get the error.

when I run the command below to check the architecture available on my M1 Mac

file /bin/bash

I get this output

/bin/bash: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]

/bin/bash (for architecture x86_64): Mach-O 64-bit executable x86_64

/bin/bash (for architecture arm64e): Mach-O 64-bit executable arm64e

I looks like I have both x86_64 and arm64e available but the error is saying

(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)

what is causing this error and how can I fix it?

Fleece

Answer 1

I found that i need to specify the architecture

so instead of

python3 make_keras_charnet_model.py

i now use this

arch -arm64 python3 make_keras_charnet_model.py

Fleece

Answer 2

I've read A LOT of information about this problem and since I've installed python3-intel64 from official Python website I have no more troubles)

Solution:

1. Go to https://www.python.org/downloads/macos/
2. Find your version of Python for project
3. Press "Download macOS 64-bit universal2 installer"
4. Install Python from archive (since that moment you can use "python3-intel64" command to run Python instead of just "python")
5. Create new venv in project from base interpreter which you installed
6. Install all requirements to venv
7. PROFIT!!!

Dibrin

tried, tested and approved – Ungenerous

spent a lot of time on this issue for my Mac M1, and so far I found it was the best solution I found. – Flameout

Answer 3

the package you are using is not compatible with the new mac hardware by the looks of it. You will need to run this using Rosetta 2, which is an apple system component.

Jezabel

Answer 4

if you install Python 3.8 or 3.9 with the macOS 64-bit universal2 installer package. you may try using python3-intel64 instead of python3.

in my MacBook M1

/usr/local/bin/python3-intel64

Grayling

Answer 5

Use the next command to install your libraries or tu run your script:

arch -x86_64 python3 -pip install [your lib]

I recommend to remove the previously installed packages to avoid the error

Schistosome

Answer 6

While the virtual env is active, I usually reinstall the package that complains with

pip install -U --force <package name>

Moccasin

Answer 7

using Rosetta

arch -arm64 python3 -m pip install theano

Junto