Conda keeps trying to install all optional dependencies?

Question

When installing Theano anaconda automatically tries to install pygpu despite this being an optional dependency. I have deleted the .theanorc file from my windows user directory.

Also when running my application Theano tries to load from the GPU. It's like it remembers somehow?

conda install theano
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment 
C:\Users\zebco\Miniconda3\envs\py35:

The following NEW packages will be INSTALLED:

libgpuarray: 0.6.9-vc14_0
pygpu:       0.6.9-py36_0
theano:      0.9.0-py36_0

Proceed ([y]/n)?

As you can see I've only specified to install theano yet conda wants to install everything including optional dependancies.

Fiesta

this is probably because this conda package is packaged with those dependencies installed. If you want it differntly you'll have to make your own conda build, or install via pip. – Hawaii

I've now tried this. Unfortunately it doesn't work. Same result – Fiesta

What makes you believe pygpu in conda is optional? How did you derive that? – Raylenerayless

@TarunLalwani you might have a point there. Am I wrong? Is it a dependency? – Fiesta

@PrimeByDesign, please see the answer I posted for details – Raylenerayless

Answer 1

Your assumption that pygpu is optional is dependent on the package manager you are using.

Regular Python (pip)

If you are using a direct Python install (obtained using brew or Python site) then you would be using pip to install theano. This basically comes from

https://pypi.python.org/pypi/Theano/1.0.0

If you download the file and unzip it. Open setup.py, you will see below lines

install_requires=['numpy>=1.9.1', 'scipy>=0.14', 'six>=1.9.0'],

So they are set as the dependencies for this package. Which means when you install theano you will also get numpy, scipy and six.

Anaconda Python (conda)

Now coming to Anaconda python. Anaconda doesn't use a package format that PyPI or pip uses. It uses its own format. In case of Anaconda you should be using conda to install the packages you need and not pip.

Conda has channels which is nothing but a repository which has some packages available. You can install a package from any channel using below

conda install -c <channel-name> <package-name>

The default channel is conda-forge. If you look at the theano package over there

https://anaconda.org/conda-forge/theano/files

And download and extract it. There will be a info/recipe/meta.yml file. You will notice below content in the same

requirements:
    build:
        - ca-certificates 2017.7.27.1 0
        - certifi 2017.7.27.1 py36_0
        - ncurses 5.9 10
        - openssl 1.0.2l 0
        - python 3.6.2 0
        - readline 6.2 0
        - setuptools 36.3.0 py36_0
        - sqlite 3.13.0 1
        - tk 8.5.19 2
        - xz 5.2.3 0
        - zlib 1.2.11 0
    run:
        - python
        - setuptools
        - six >=1.9.0
        - numpy >=1.9.1
        - scipy >=0.14
        - pygpu >=0.6.5,<0.7

Which specifies that if you want to run this package then pygpu is also on of its dependencies. So conda downloads pygpu as a dependency which you though was optional (which is probably true if you were using regular python and pip)

Raylenerayless

Answer 2

Update:
Usually, 'Optional Dependency' is an oxymoron. Something optional is not a dependency, a dependency is a software package another piece of software depends on to function for features.
One may get by without a dependency if the dependency does not interact with the package except for one atomized feature which is not being used. As a beginner I would suggest you not take this path.

I am not super familiar with Theano, but Theano can use the system's GPU to speed up its computations, and it seems to me pygpu and gpulibarray are what enable this functionality. Which means it is not optional.

I believe pygpu is 'optional' if you do not wish to use the GPU for speeding up computation (only done if the GPU is powerful enough to be useful for this).

The --no-deps command above allows you to install a package without its dependencies but that is rarely wise, unless one really knows what they are doing. As a beginner I would not recommend you go down this path yet. Conda was designed specifically to ensure scientific packages are easily managed with all necessary stuff installed without any fuss or muss. pip is a general python package manager, but is not built specifically for scientific packages.

If you wish to install theano without installing its dependencies, then you have one of three options:

1. use conda install theano --no-deps.
2. Install it using pip instead of conda, using pip install theano. This will install theano, numpy, scipy and six but not pygpu and libgpuarray.
3. Create a custom conda build file for Theano. Documentation is at:
   https://conda.io/docs/user-guide/tasks/build-packages/index.html

Original Answer:

You probably know this already but, use this command instead:

conda install theano --no-deps

This does not install dependencies of the package. If you already have the essential dependencies installed, as it would seem, this should work out for you.

libgpuarray is a dependency of pygpu. With this command switch neither will be installed.

Can you share the .yaml file that you edited?

Tinishatinker

I didn't edit any yaml file. No I didn't know the above I am a beginner. I just can't see why conda is trying to install pygpu. I don't think it's a dependency. – Fiesta

What makes you think it is not? 'Optional dependency' is an oxymoron. How can something be a dependency if it is optional? Also, you mentioned in comment to @maarten fabre "I've now tried this. Unfortunately it doesn't work. Same result.", when he says you have to create your own conda build. How did you create your own conda build without editing a yaml file? – Tinishatinker

I am not familiar with the intricacies of Theano, but Theano enables the use of GPU to speed up data processing and pygpu, libgpuarray seem like they are what enable this feature, which means it is not optional. – Tinishatinker

@kchawla-pi. It's not that complicated. Optional dependencies are needed for optional features. If you have the dependency, you get the feature, such as GPU speedup. If you don't have the dependency you have to use CPU instead, but everything else still works. Many packages have these sort of optional features, and package managers should support them. – Saurel

Thanks @BenFarmer . What you say is correct. I didn't know it at the time. – Tinishatinker