Installing scipy and scikit-learn on apple m1

Question

The installation on the m1 chip for the following packages: Numpy 1.21.1, pandas 1.3.0, torch 1.9.0 and a few other ones works fine for me. They also seem to work properly while testing them. However when I try to install scipy or scikit-learn via pip this error appears:

ERROR: Failed building wheel for numpy

Failed to build numpy

ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly

Why should Numpy be build again when I have the latest version from pip already installed?

Every previous installation was done using python3.9 -m pip install ... on Mac OS 11.3.1 with the apple m1 chip.

Maybe somebody knows how to deal with this error or if its just a matter of time.

Alderson

pip has a --no-use-pep517 option. See if that works. – Blessed

thanks for the hint, are there any downsides using this flag for installation? – Alderson

Answer 1

UPDATE: scikit-learn now works via pip ✅

Just first brew install openblas - it has instructions for different processors (wikipedia)

brew install openblas
export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
export CFLAGS="-falign-functions=8 ${CFLAGS}"
# ^ no need to add to .zshrc, just doing this once.
pip install scikit-learn

Worked great on Apple Silicon M1 🎉

Extra details about how Pip works

Pip downloaded the source from Pipy, then built the wheel targeting MacOS X 12.0, and arm64 (apple silicon): scikit_learn-1.0.1-cp38-cp38-macosx_12_0_arm64.whl.

Building wheels for collected packages: scikit-learn
  Building wheel for scikit-learn (pyproject.toml) ... done
  Created wheel for scikit-learn: filename=scikit_learn-1.0.1-cp38-cp38-macosx_12_0_arm64.whl size=6364030 sha256=0b0cc9a21af775e0c8077ee71698ff62da05ab62efc914c5c15cd4bf97867b31
Successfully built scikit-learn
Installing collected packages: scipy, scikit-learn
Successfully installed scikit-learn-1.0.1 scipy-1.7.3

Note on Pipy: we usually download either a pre-built wheel (yay, this is excellent for reliable distribution and ensuring compatability). Or, if no prebuilt wheel exists (sad) then we download a tar.gz and build it ourselves. This happens because the authors don't publish a prebuilt wheel to Pipy, but more and more people are adding this to their CI (github actions) workflow. Building the wheel ourselves takes more cpu time, and is generally less reliable but works in this case.

Here we are downloading a pre-built wheel that has very few limitations: it works for any version of python 3, for any os, for any architecture (like amd64 or arm64): click-8.0.3-py3-none-any.whl

Collecting click>=7.0
  Downloading click-8.0.3-py3-none-any.whl

Here apparently we had no wheel available, so we have to build it ourselves with setuptools running setup.py.

Collecting grpcio>=1.28.1
  Downloading grpcio-1.42.0.tar.gz (21.3 MB)
     |████████████████████████████████| 21.3 MB 12.7 MB/s
  Preparing metadata (setup.py) ... done

## later in the process it installs using setuptools 
Running setup.py install for grpcio ... done

Good luck and happy piping.

Unwashed

Worked fine for me, even though I didn't install openblas with brew separately. With a bit of waiting, the wheel was successfully built with pep 517. Sklearn was also tested with a simple tutorial to confirm working installation. Thanks for your answer. – Alderson

Do you know if this works for building from source? Say if you are making a change locally to the scikit-learn code. – Collimator

@Collimator Yes, this will work for building from source, because that's essential the same as what we're dong when we're building the wheel for scikit-learn. Good luck! – Unwashed

couldn't install scikit-learn==0.24.1 for m1 pro, but ok with 1.1 – Remains

Hi @SolomidHero, yes I'm having the same issue with scikit-learn==0.24.2, it seems that older versions of scikit-learn is requiring platform_machine variable aarch64 and thus it didn't work on apple M1? – Thousandth

Thanks for commenting. I updated the solution so future people won't experience your difficulties. – Unwashed

version 1.x worked on m1, but it's painful, scikit-learn 1.x version is imcompatible with older version 0.xxx. – Simulant

Answer 2

Finally managed to sort this using below steps:

1. install python 3.9 using brew
 (brew install [email protected])
2. setup PATH to pick brew python3 first (export PATH=opt/homebrew/bin:$PATH)
3. Run below commands to install scipy and scikit-learn

    >> /opt/homebrew/bin/brew install openblas
    >> export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
    >> export CFLAGS="-falign-functions=8 ${CFLAGS}"
    >> git clone https://github.com/scipy/scipy.git
    >> cd scipy
    >> git submodule update --init
    >> /opt/homebrew/bin/pip3 install .
    >> /opt/homebrew/bin/pip3 install scikit-learn

4. go to the folder where virtual env needs to be created and run python3 -m venv --system-site-packages .venv

Scolopendrid

This is the easiest way I've found to get scipy installed properly. I feel this comment should get more attention as most guides you find when searching on how to install scipy go through conda forge. – Choline

how to do your number 2? – Ashjian

@EnzoY export PATH=opt/homebrew/bin:$PATH – Scolopendrid

Answer 3

As of March 14 (pie day!) 2022, all I had to do was type the following in a conda environment:

conda install --channel=conda-forge scikit-learn

That was it. I am running a MacBook Pro (13", M1, 2020) with MacOS 11.6, and python 3.8 in the conda environment.

Kwakiutl

This worked perfectly after an hour of searching. – Ungual

This works best if the conda installation is also Apple silicon distribution – Punic

Answer 4

Please see this note of scikit-learn about

Installing on Apple Silicon M1 hardware

The recently introduced macos/arm64 platform (sometimes also known as macos/aarch64) requires the open source community to upgrade the build configuation and automation to properly support it.

At the time of writing (January 2021), the only way to get a working installation of scikit-learn on this hardware is to install scikit-learn and its dependencies from the conda-forge distribution, for instance using the miniforge installers:

https://github.com/conda-forge/miniforge

The following issue tracks progress on making it possible to install scikit-learn from PyPI with pip:

https://github.com/scikit-learn/scikit-learn/issues/19137

Hydrozoan

Thanks for you're reply. Already read this instructions, but since they are dated to January 21 I thought there might be a fix / better way of installing the packages. – Alderson

I can't tell for sure if there are any alternatives. But when you follow the GitHub link for issue #19137, you can see that it is still open and currently 1 of 3 tasks have been completed. They are actively working on it, but it is not resolved yet. – Hydrozoan

Update: scikit-learn is now AVAILABLE VIA PIP. Just install openblas first, see my answer: https://mcmap.net/q/263570/-installing-scipy-and-scikit-learn-on-apple-m1 – Unwashed

Answer 5

Apple M1 Pro, macOS 12.1

Python 3.9.10 (main, Jan 15 2022, 11:40:53) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
pip 22.0.3

install numpy and scipy

brew install openblas gfortran
OPENBLAS="$(brew --prefix openblas)" pip install numpy==1.19.3
OPENBLAS="$(brew --prefix openblas)" pip install scipy==1.7.2

install scikit-learn (0.21.3)

pip install cython
brew install libomp
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
export CPPFLAGS="$CPPFLAGS -Xpreprocessor -fopenmp"
export CFLAGS="$CFLAGS -I/opt/homebrew/Cellar/libomp/13.0.1/include"
export CXXFLAGS="$CXXFLAGS -I/opt/homebrew/Cellar/libomp/13.0.1/include"
export LDFLAGS="$LDFLAGS -L/opt/homebrew/Cellar/libomp/13.0.1/lib -lomp"
export DYLD_LIBRARY_PATH=/opt/homebrew/Cellar/libomp/13.0.1/lib
pip install scikit-learn==0.21.3

Regin

Answer 6

I do not know why numpy would be built again (maybe scipy has other version requirements?), but I can report on a way how to install scipy and scikit-learn. I just had success using the following sequence of commands, base on Python 3.8.11 (especially scipy compilation took some time, so this process is not a quick one, unfortunately):

# SciPy:
python -m pip install --no-cache --no-use-pep517 pythran cython pybind11 gast"==0.4.0"
pyenv rehash
python -m pip install --no-cache --no-binary :all: --no-use-pep517 scipy"==1.7.1"

# Scikit-Learn
python -m pip install --no-use-pep517 scikit-learn"==0.24.2"

So the information on the scikit-learn homepage about the conda installation path being necessary is outdated, it seems.

There were the Apple developer tools (XCode, xcode-select) and some other packages installed via homebrew, and in a pyenv virtualenv environment before, which might also be relevant, so I add them here below:

brew install openblas [email protected] pkg-config pyenv pyenv-virtualenv
python -m pip install numpy==1.19.5

I needed the older numpy version because there is also tensorflow installed in this environment which is not compatible with newer numpy versions; I do not know whether your newer numpy could contribute to the problems you encountered.

It appears that there is a lot currently changing with respect to Apple M1 support, so this might become still easier soon.

Brandie

Answer 7

I guess Scipy and Numpy is not directly supported on arm64 processor (MAC M1 OS BigSur and higher versions) which requires the users to manually install and build the dependencies. I figured a work around by creating a virtual environment with Python version 3.8. Once the final releases are rolled out, this changes can be removed easily without any conflict.

1. Setup a python virtual environment with Python version 3.8.

python3.8 -m venv <environment_name>

2. Activate the environment

source <environment_name>/bin/activate

3. Install numpy and scipy

pip3 install numpy
pip3 install scipy

Balustrade

Answer 8

I'm chiming in here since I'm installing all of this using a Docker container. Host machine is M1 MBP running Monterey 12.0.1 - I found the following solution in this thread on GitHub. It seems the scipy team will have an update soon.

Make sure you are on Python 3.9 and using HomeBrew, run brew install openblas. Once that is done you can add the following lines to your Dockerfile:

RUN OPENBLAS="/opt/homebrew/opt/openblas" CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install scipy
RUN OPENBLAS="/opt/homebrew/opt/openblas" CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install scikit-learn

Savannasavannah

Answer 9

The best way to do this is using Miniforge . Confirmed workiing for Macbook M1 and OS 12(monterey)

1. On macbook M1, you can install miniforge using the command brew install miniforge.

2. Once this is installed, you can create a virtual environment using the command conda create -n .venv python

3. activate the environment using the command conda activate .venv

4. Install scikit-learn to the environment using conda install scikit-learn

The official issue tracker related to the scikit-learn installation is in their github issues . Some of the solutions suggested there is working for Mac OS 11, but not for Mac OS 12.
I finally found the working solution here

Hypno

Answer 10

I solved this by using poetry and setting python restricted dependencies scipy = { version = "^1.8.0", python = "~3.9" } in pyproject.toml.

Frogmouth

Answer 11

I face the same problem on macOS 11.6.8 (BigSur) with the Apple M1 chip.

After much research, I found that scipy now supports the Apple M1 chip, but it requires macOS 12.

So, to install scipy on the Apple M1 chip, you can do follow 2 ways bellow:

1. Update your macOS to 12.
2. Use Conda to manage Python virtual environment and install scipy by

conda activate myenv
conda install scipy

I hope 2 ways above can help you!

Husbandry