Installing Tensorflow on macOS on an Arm MBP

Question

When attempting to install tensorflow on an ARM M1 MacBookPro, I am seeing the following issues:

% pip install tensorflow
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow

After searching around, I found a package called tensorflow-macos which produces the following issues:

clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly
    error: Command "gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNO_ATLAS_INFO=3 -DHAVE_CBLAS -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/umath -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/npymath -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/common -Inumpy/core/include -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/include/numpy -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/Users/thing/dev/SOTAMoon/venv/include -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/common -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/npymath -c numpy/core/src/multiarray/descriptor.c -o build/temp.macosx-10.9-universal2-3.9/numpy/core/src/multiarray/descriptor.o -MMD -MF build/temp.macosx-10.9-universal2-3.9/numpy/core/src/multiarray/descriptor.o.d -faltivec -I/System/Library/Frameworks/vecLib.framework/Headers -std=c99" failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/a7/81/20d5d994c91ed8347efda90d32c396ea28254fd8eb9e071e28ee5700ffd5/h5py-3.1.0.tar.gz#sha256=1e2516f190652beedcb8c7acfa1c6fa92d99b42331cbef5e5c7ec2d65b0fc3c2 (from https://pypi.org/simple/h5py/) (requires-python:>=3.6). Command errored out with exit status 1: /Users/thing/dev/SOTAMoon/venv/bin/python3 /private/var/folders/8k/z291bhgd5gs06tp9b4j6_bb40000gn/T/pip-standalone-pip-8qa87uy0/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /private/var/folders/8k/z291bhgd5gs06tp9b4j6_bb40000gn/T/pip-build-env-bxy_nyoo/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'numpy==1.12; python_version == "3.6"' 'numpy==1.19.3; python_version >= "3.9"' 'numpy==1.14.5; python_version == "3.7"' pkgconfig 'Cython>=0.29; python_version < "3.8"' 'Cython>=0.29.14; python_version >= "3.8"' 'numpy==1.17.5; python_version == "3.8"' Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement h5py~=3.1.0 (from tensorflow-macos) (from versions: 2.2.1, 2.3.0b1, 2.3.0, 2.3.1, 2.4.0b1, 2.4.0, 2.5.0, 2.6.0, 2.7.0rc2, 2.7.0, 2.7.1, 2.8.0rc1, 2.8.0, 2.9.0rc1, 2.9.0, 2.10.0, 3.0.0rc1, 3.0.0, 3.1.0, 3.2.0, 3.2.1, 3.3.0)
ERROR: No matching distribution found for h5py~=3.1.0

Unfortunately I couldn't work out a solution to the 'faltivec' issue. I also found a tensorflow package made by Apple that seems specifically geared for M1's, however going by this tutorial produces the following issues:

% pip install --upgrade --force --no-dependencies https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl
ERROR: tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl is not a supported wheel on this platform.

After this, I am a bit stuck. I have upgraded pip to 21.1.3, and my python version is Python 3.9.6.

Rawboned

Follow the instructions mentioned in the Tensorflow doc to install Tenosrflow. If you are facing version incompatibility, build Tensorflow from source by following the instructions mentioned here. Thanks! – Ecumenism

Answer 1

Tensorflow-macos and Tensorflow-metal Install

Currently, to harness the M1 GPU you need to install Tensorflow-macos and TensorFlow-metal as opposed to Tensorflow, the install steps are detailed here, they can be summarized as follows using mini-forge:

conda create -n tf python=3.8 -y
conda activate tf
conda install -c apple tensorflow-deps -y
# Navigate the issue with conda environments
# built against pre-macOS 11 SDK use (Just in case)
# this solves the Intel / M1 install issues:
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-macos 
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-metal

to test all is well we can run:

#!/usr/bin/env bash

PYCMD=$(cat <<EOF
import tensorflow as tf

tf.config.list_physical_devices()
with tf.device('/GPU'):
  a = tf.random.normal(shape=(2,), dtype=tf.float32)
  b = tf.nn.relu(a)

print(a)
print(b)

EOF
)

python3 -c "$PYCMD" 2>/dev/null

the expected output for my test machine:

>./val.sh
Metal device set to: Apple M1

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

tf.Tensor([-0.4044231   0.08157467], shape=(2,), dtype=float32)
tf.Tensor([0.         0.08157467], shape=(2,), dtype=float32)

ok, everything is (assumed) to be working...

Side-Note: Package Install Issues:

One thing to note is that that packages that rely on tensorflow will try to install tensorflow as they will likely not have an explicit dependency on tensorflow-macos or tensorflow-metal.

When you install tensorflow dependent packages or code you'll likely have to review the requirements.txt file before running:

python3 pip install -r requirements.txt

The requirements.txt will try to install the tensorflow package, not the tensorflow-macosx or tensorflow-metal packages which will result in a host of issues if you run the requirements.txt file unchanged. That and explicit versions of packages == which can also cause problems.

Hopefully, this points you in the right direction in thinking about the hurdles involved in getting python to work with the Apple-M1 laptops using the TensorFlow metal plugin to harness the M1 GPU features.

Stay safe and well.

Shetler

Answer 2

try

python3 -m pip install tensorflow-macos
python3 -m pip install tensorflow-metal

you may also need:

brew install hdf5
export HDF5_DIR=$(brew --prefix hdf5) 

Wangle