How to install h5py (needed for Keras) on MacOS with M1?

Question

I have an M1 MacBook. I have installed python 3.9.1 using pyenv, and have pip3 version 21.0.1. I have installed homebrew and hdf5 1.12.0_1 via brew install hdf5.

When I type

pip3 install h5py

I get the error:

Requirement already satisfied: numpy>=1.19.3 in /Users/.../.pyenv/versions/3.9.1/lib/python3.9/site-packages (from h5py) (1.20.0)
Building wheels for collected packages: h5py
  Building wheel for h5py (PEP 517) ... error

  Loading library to get build settings and version: libhdf5.dylib
  error: Unable to load dependency HDF5, make sure HDF5 is installed properly
  error: dlopen(libhdf5.dylib, 6): image not found
  ----------------------------------------
  ERROR: Failed building wheel for h5py

I saw that libhdf5.dylib is present in /opt/homebrew/opt/hdf5/lib, so I tried export LDFLAGS="-L/opt/homebrew/opt/hdf5/lib" and export CPPFLAGS="-L/opt/homebrew/opt/hdf5/include" beforehand but they don't help.

How can I install h5py?

I am actually installing h5py as a requirement to install Keras.

Thanks!

Unglue

I'm not a Mac guy, so of limited help. However, I saw a similar question about installing Python packages on Mac M1. He did NOT use pip to install packages. Instead, he used macports, like this: sudo port install package_name. Good luck – Titanesque

Answer 1

This works for me:

$ brew install hdf5
$ export HDF5_DIR="$(brew --prefix hdf5)"
$ pip install --no-binary=h5py h5py

Dialysis

docs.h5py.org/en/latest/build.html#custom-installation for more info – Folketing

Happy to hear this worked for others. This failed for me however with 6 compilation errors (h5py version 2.10.0--Keras compatability). – Marcellus

it might not be 1.12.0_4 anymore. For me it was 1.13.0. Update accordingly – Heraclid

Assigning a version tracker with VERSION=$(brew list --versions hdf5 | cut -d' ' -f2) after installing hdf5 allows you to select the right directory as in export HDF5_DIR=/opt/homebrew/Cellar/hdf5/$VERSION – Simonnesimonpure

export HDF5_DIR=$(brew --prefix hdf5) is a much simpler way to get the directory. – Darr

I also had to export the include directory: bash export HDF5_DIR="$(brew --prefix hdf5)" export CPATH=$(brew --prefix)/include:${CPATH} – Nieman

This works for me on a brand new M1 Mac, and got me on the path to installing Tensorflow which is NOT easy on the M1 Mac. I had to pip install tensor flow-metal followed by pip install tensorflow-macos. BUT when I did this, tensorflow-macos installation would fail building the HDF5 stuff. And your post worked and then I got my Tensorflow demo running. – Howrah

It cannot hep. Here is the issue "error: dlopen(/usr/local/opt/hdf5/lib/libhdf5.dylib, 0x0006): tried: '/usr/local/opt/hdf5/lib/libhdf5.dylib' (mach-o file, but is an incompatible architecture (have (x86_64)," – Chapell

Answer 2

Try to install h5py using conda. Works for me.

conda install h5py

Monjo

Easy and straightforward solution. Works on my Apple M1 Mac Mini. Thanks! – Ronaldronalda

worked on a MacBook with M1 – Cheke

Happy that Anaconda is there for us M1 folks. They do a pretty good job of getting everything working. Sadly, I can't use Anaconda for every project. – Marcellus

This works great. I'm not even using Anaconda (and the environment I used this for still is not). This installed h5py and libs onto the system and satisfied the requirement for all my python.org/pip managed environments for tables and other packages as well. I wanted to avoid getting into homebrew on this system, and this was perfect! – Canzone

Answer 3

Here are some steps you can follow to install hdf5 on macOS M1.

Step 0 (optional): Uninstall hdf5 with brew if you have installed.

brew uninstal hdf5 --ignore-dependencies  

Step 1: Install hdf5 with brew.

arch -arm64 brew install hdf5

Step 2: Find the directory of hdf5 and hdf5.h.

sudo find / -iname "*hdf5.h*" # use sudo if possible

On my laptop, the output is

/usr/local/include/hdf5.h
/usr/local/Cellar/hdf5/1.12.2_2/include/hdf5.h
/usr/local/Cellar/opencv/4.7.0_2/include/opencv4/opencv2/flann/hdf5.h
/usr/local/Cellar/vtk/9.2.6/include/vtk-9.2/vtk_hdf5.h
/System/Volumes/Data/usr/local/include/hdf5.h
/System/Volumes/Data/usr/local/Cellar/hdf5/1.12.2_2/include/hdf5.h
/System/Volumes/Data/usr/local/Cellar/opencv/4.7.0_2/include/opencv4/opencv2/flann/hdf5.h
/System/Volumes/Data/usr/local/Cellar/vtk/9.2.6/include/vtk-9.2/vtk_hdf5.h

Step 3: Set CPATH and HDF5_DIR.

export CPATH=$CPATH:/usr/local/include/:/usr/local/Cellar/hdf5/1.12.2_2/include

and

export HDF5_DIR=/usr/local/Cellar/hdf5/1.12.2_2

Note that on your laptop, these directories can be different.

Step 4: Install hdf5 with pip.

arch -arm64 pip3.9 install hdf5

It then successfully completes the installation.

Chapell