Trouble installing Pandas on new MacBook Air M1

Question

I recently got a new MacBook Air M1, but I’m having trouble installing Pandas, since prebuilt binaries are not available for macOS/arm64. How can I install Pandas on an Apple Silicon Mac?

Argonaut

I shouldn't have to ask (i.e., this should already be in your question), but what's the trouble? – Premedical

Answer 1

Maybe it is too late. But the only solution worked for me is installing from source if you do not want to use rosetta2 or moniconda

python3 -m pip install virtualenv
virtualenv -p python3.8 venv
source venv/bin/activate
pip install --upgrade pip
pip install numpy cython
git clone --depth 1 https://github.com/pandas-dev/pandas.git
cd pandas
python3 setup.py install

Richter

This did not work for me. I'm running Big Sur on an M1 Mac and I got the error "Failed to build numpy. ERROR: Could not build whells for numpy which use PEP 517 and cannot be installed directly." – Mimicry

I think for python3.8, you should not have this error. For python3.9, tho you just use the hint from the error, install from source git clone github.com/numpy/numpy.git && cd numpy && python3 setup.py install – Richter

Genius, saved my day also for numpy and scikit learn for Django, thanks – Imparisyllabic

Thanks this totally helped, spent 4 and 1/2 hours trying to fix this problem. Much blessed for this knowledge. – Galle

Dis not work for me, still got the same error: venv/lib/python3.8/site-packages/pandas-1.4.0.dev0+264.g9c0a1ebec6-py3.8-macosx-11.4-arm64.egg/pandas/_libs/interval.cpython-38-darwin.so: mach-o, but wrong architecture – Recording

this did work for me on a new Mac Book Pro running Monterey on M1 architecture. I am using pyenv. First, installed latest python 3.8. Then created a virtual environment from that. In that virtual environment I ran the installs and build above. – Ina

@Mimicry Re: PEP517 - see https://mcmap.net/q/263570/-installing-scipy-and-scikit-learn-on-apple-m1 – Dialogist

Answer 2

What works for me:

pip3 install cython
OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 pip3 install numpy --no-use-pep517
OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 pip3 install pandas --no-use-pep517

Keese

I am in arm64 mode. This did not work with Python 3.9.x, but it DID work with Python 3.8.11. Thank you. – Compare

Worked for me using Python 3.9.x trying to install numpy==1.20.0 which was required for scikit-learn. I had earlier tried with the OPENBLAS export thingy but it did not work. What worked was adding --no-use-pep517 along with installing cython. – Eogene

Answer 3

There are two methods to do this. But the underlying process is the same in both.

Method-1

Open a terminal window. Run arch. The result should be arm64. Now do arch -x86_64 zsh. This will create a x86 emulated zsh session inside the terminal. Now if you run arch command, the result should be i386.

Now do pip install pandas or pip3 install pandas (whichever command has the correct version). And this will work. Perform exit command to exit the emulated zsh session.

You can also execute a single command under x86 emulation without creating an emulated zsh session. Run arch -x86_64 pip install pandas. This will run pip command under x86_emulation but the terminal stays in the native arm environment.

---

Method-2

Open Terminal App using Rosetta 2 which emulates a x86 environment. Then do pip install pandas or pip3 install pandas (whichever command has the correct version) And that should work.

This trick would work for almost all the python packages that fails to install when tried in ARM architecture

---

Opening Terminal app using Rosetta 2

1. Close the Terminal Application
2. Find the Terminal App in Finder (usually located in Macintosh HD/Applications/Utilities)
3. Secondary Click the Terminal Icon >> Get Info
4. Check the checkbox labeled Open Using Rosetta
5. Now launch a Terminal window (The new Terminal window will be opened using Rosetta emulation)

---

Edit-1: If you install a package under x86 emulation, you have to run your python code also under x86 emulation. If you try to run python in the native ARM environment, then architecture mismatch error will be thrown by the package for which you installed the x86 version.

Kiethkiev

arch -arm64 pip install pandas or arch -arm64 pip install -r requirements.txt or arch -arm64 zsh && poetry install – Serpent

This depends on how your codebase is setup – Kiethkiev

Thank you! arch -x86_64 pip3 install pandas did the trick! – Achromatize

As of May 2022, after a lot of trial and error I finally had to just swallow my pride and avoid arm. I have been trying to install numpy to no avail. – Nanon

Been there, done that! – Kiethkiev

Answer 4

Most of these articles fail to address the issue of 'not' wanting to run under Rosetta. Apple's TensorFlow fork works in native M1 mode, but there is no Pandas to match that. If you choose to run under Rosetta then you won't be able to find a matching TensorFlow. It's a lose lose situation.

Ullund

Not true if you are using (mini)conda. You can use conda install which does seem to work. – Riptide

Answer 5

You can just uset the --no-use-pep517 flag, and you can install pandas with no problems.

Here is how I installed on my laptop:

~/Documents/projects/m-proj                                                                                                                                               
❯ pip install cython --no-use-pep517 
Collecting cython
  Using cached Cython-0.29.22-py2.py3-none-any.whl (980 kB)
Installing collected packages: cython
Successfully installed cython-0.29.22

~/Documents/projects/m-proj                                                                                                                                               
❯ pip install numpy --no-use-pep517
Collecting numpy
  Using cached numpy-1.20.1.zip (7.8 MB)
Using legacy 'setup.py install' for numpy, since package 'wheel' is not installed.
Installing collected packages: numpy
    Running setup.py install for numpy ... done
Successfully installed numpy-1.20.1

~/Documents/projects/m-proj                                                                                                                                           
❯ pip install pandas --no-use-pep517
Collecting pandas
  Using cached pandas-1.2.3.tar.gz (5.5 MB)
Collecting python-dateutil>=2.7.3
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Requirement already satisfied: pytz>=2017.3 in ./venv-trans/lib/python3.9/site-packages (from pandas) (2021.1)
Requirement already satisfied: numpy>=1.16.5 in ./venv-trans/lib/python3.9/site-packages (from pandas) (1.20.1)
Requirement already satisfied: six>=1.5 in ./venv-trans/lib/python3.9/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
Using legacy 'setup.py install' for pandas, since package 'wheel' is not installed.
Installing collected packages: python-dateutil, pandas
    Running setup.py install for pandas ... done
Successfully installed pandas-1.2.3 python-dateutil-2.8.1

Indigenous

This works for me with numpy-1.20.1 but not with numpy-1.20.2 which is required for python3.9. A bit of an impass, that seems super time dependent as right now I can install python3.9 in on M1 (arch arm64) with pyenv, but not python3.8. It would be nice to get python and pandas on arm64. – Hypertonic

This worked for my with m2 mbp and numpy~=1.19 pandas~=1.1 – Vullo

Answer 6

Miniconda is by far the simplest option.

brew install --cask miniforge
conda create -n myenv python=3.8
conda init zsh
conda activate
conda install pandas

If you need an older version (not available for arm64 in conda-forge)

brew install --cask miniforge
conda create -n myenv python=3.8
conda init zsh
conda activate
pip install cython
curl -L https://github.com/pandas-dev/pandas/archive/v1.1.2.zip --output pandas.zip  
unzip pandas.zip     
cd pandas-1.1.2 
python3 setup.py install
rm pandas.zip
rm -rf pandas-1.1.2

Cockney

Answer 7

1. Install conda miniforge for OS X arm64(Apple Silicon)

https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh

2. Then create virtual environment and install your required package conda create -n myenv python=3.6 conda activate myenv conda install numpy conda install pandas

Essen

Great. I think Miniforge is the way to go. Was reading Pandas in Action and their advice was to go with Anaconda. Anaconda Apple Silicon page is just way too confusing! – Submersible

Answer 8

Check out this article Apple M1 — Matplotlib, Python, and Jupyter Lab or this one (same author) Apple M1, Python, Pandas, and Homebrew

Maybe you just need run terminal with Rosetta (the second article indicates this)

Michealmicheil

The article basically just says tick "Start with Rosetta" for iTerm2. That didn't help me. – Hulett

Update March 8-2022: Macbook with M1 and MacOS Monterey. The approach that worked for me still was opening zsh in rosetta mode and following the regular installation procedures. The virtual-env approach wasn't still working out for me the – Jahdai

Answer 9

So I found the solution. Apparently even though I checked the box to open the term using Rosetta, it didn't do it the first time. I finally discovered this by executing "arch" on the term, which returned "arm64", not "i386". Rebooting apparently made the "rosetta" selection stick, and I was then able to "pip3 install pandas" successfully. Of course it would have been nicer to install it in native arm mode. Thanks to all for the pointers.

Magdalenamagdalene

Different, but related question: do you have any advice on how to install tensorflow? "pip3 install storage.googleapis.com/tensorflow/mac/cpu/…" returns "ERROR: tensorflow-2.4.0-cp38-cp38-macosx_10_14_x86_64.whl is not a supported wheel on this platform." – Tattle

Answer 10

reference this page https://ittone.ma/ittone/python-install-pandas-on-mac-m1-big-sur-into-multiple-virtualenv/

1. pip install cython
2. git clone https://github.com/numpy/numpy.git
3. cd numpy
4. python3 setup.py install
5. git clone https://github.com/pandas-dev/pandas.git
6. cd pandas
7. python3 setup.py install

Chisel

When I tried these steps, everything worked until step 7 when I receive an error: ModuleNotFoundError: No module named 'numpy' – Irenairene

Answer 11

I noticed that many of the users including myself ran into this issue when installing pandas while also using a virtual environment while also on an m1 computer.

The approach that worked for me after the top answer by ropoma failed involved uninstalling pandas, and then ensuring that the pip package manager specified by the (env) is used to install rather than from elsewhere:

pip3 uninstall pandas
python3 -m pip install pandas

Mithridatism

Answer 12

I kept running into problems with python on my M1 Mac until I went completely to Rosetta on the command line. For that, I did the following:

1. Update Rosetta: In a Terminal type:

softwareupdate --install-rosetta

2. In Finder, type ⇧⌘G and go to /Applications/Utilities. Then duplicate Terminal:

3. Rename the second Terminal to "Rosetta" (or whatever you like) and have it execute in Rosetta by checking "Open using Rosetta" in the "Get Info" dialogue:

4. Open a Rosetta Terminal and make sure it shows i386 when you issue the command arch:

4. In that terminal, install homebrew (per the homebrew homepage):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

5. Once homebrew has been installed, install miniconda using homebrew:

brew install --cask miniconda

6. Create a conda environment, for instance here a python 3.9 env named pandas:

conda create -n pandas python=3.9

7. Activate the environment:

conda activate pandas

From here on out, you have a fully functioning i386 Python system. This has resolved all problems that I had with Pandas, Numpy, Azure, etc. on my M1 Mac.

Stutter