Conda UnsatisfiableError even when seemingly there isn't any conflict

Question

For example, I have the following environment.yml file

name: harmonic-backend
channels:
  - conda-forge
dependencies:
  - python==3.8.10
  - apache-beam==2.31.0
  - pyarrow==4.0.1

prefix: /opt/homebrew/Caskroom/miniforge/base/envs/harmonic-backend

And following is the output when I try to create an environment accordingly

sumit@Sumits-MacBook-Air backend % conda env create
Collecting package metadata (repodata.json): done
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
apache-beam==2.31.0 -> crcmod[version='>=1.7,<2.0'] -> python[version='3.8.*|>=2.7|>=3|>=3.6|>=3.9,<3.10.0a0']
python==3.8.10
apache-beam==2.31.0 -> python[version='>=3.8,<3.9.0a0']

Package pyarrow conflicts for:
apache-beam==2.31.0 -> pyarrow[version='>=0.15.1,<5.0.0']
pyarrow==4.0.1

Package setuptools conflicts for:
python==3.8.10 -> pip -> setuptools
apache-beam==2.31.0 -> grpcio[version='>=1.29.0,<2'] -> setuptools

Package ncurses conflicts for:
pyarrow==4.0.1 -> python[version='>=3.9,<3.10.0a0'] -> ncurses[version='>=6.2,<6.3.0a0']
python==3.8.10 -> readline[version='>=8.1,<9.0a0'] -> ncurses[version='>=6.2,<6.3.0a0']

Looking at the output, I can't find any conflict. What's going on here and how do I solve this?

For more context, I'm using miniforge installed using brew on an M1 mac.

Maritime

How important is it that you have exactly those versions of each package? – Rosenkranz

@MattThompson from my analysis, loosening versions wouldn't help. All apache-beam builds for osx-arm64 platform depend on dill >=0.3.1.1,<0.3.2 but there is no such package. – Villainage

Answer 1

Fixed

This is now working thanks to the efforts of one of the apache-beam-feedstock maintainers (see Issue).

For future reference, when such failures are observed please report them to the Conda Forge feedstock maintainers. Many of the builds forr osx-arm64 are compiled and packaged, but there is not any CI infrastructure set up to test them. Hence, the maintainers are dependent on end-users in the community raising issues to them when them come up on native osx-arm64 hardware.

Otherwise, if fixes aren't immediately implemented, Apple Silicon users should consider a containerized solution (e.g., Docker) that doesn't use the osx-arm64 platform. Or use Rosetta to emulate osx-64 platform.

---

Analysis of Failure (Outdated)

Mamba gives a more interpretable result (assuming you are on native osx-arm64 platform):

$ CONDA_SUBDIR=osx-arm64 mamba create -n foo python=3.8.10 apache-beam=2.31.0 pyarrow=4.0.1

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.16.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['python=3.8.10', 'apache-beam=2.31.0', 'pyarrow=4.0.1']

conda-forge/noarch       Using cache
bioconda/noarch          Using cache
pkgs/main/noarch         [====================] (00m:00s) No change
pkgs/r/osx-arm64         [====================] (00m:00s) 404 Failed
pkgs/r/noarch            [====================] (00m:00s) No change
pkgs/main/osx-arm64      [====================] (00m:00s) Done
bioconda/osx-arm64       [====================] (00m:00s) 404 Failed
conda-forge/osx-arm64    [====================] (00m:00s) Done
Encountered problems while solving:
  - nothing provides dill >=0.3.1.1,<0.3.2 needed by apache-beam-2.31.0-py38hea4295b_0

Namely, the dill package that apache-beam=2.31.0 wants isn't available for osx-arm64 platform. The underlying issue is that dill isn't required to build the apache-beam package, and since Azure doesn't provide native M1 images, the Conda Forge pipeline never tests whether osx-arm64 builds are actually installable(!).

Villainage