Login/Register
Creating Python package which uses Cython: ValueError
Asked Answered
pythoninstallationcythonsetuptoolssetup.py
L

1

6

My problem:

I am trying to create a python package to upload it to PyPI.

However, when I run

python -m build

inside the project directory (example_package), then I get the following output and error:

WARNING: Skipping example_package as it is not installed.
* Creating venv isolated environment...
* Installing packages in isolated environment... (Cython, numpy, setuptools, wheel)
* Getting dependencies for sdist...
running egg_info
creating example_package.egg-info
writing example_package.egg-info/PKG-INFO
writing dependency_links to example_package.egg-info/dependency_links.txt
writing top-level names to example_package.egg-info/top_level.txt
writing manifest file 'example_package.egg-info/SOURCES.txt'
reading manifest file 'example_package.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'example_package.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing example_package.egg-info/PKG-INFO
writing dependency_links to example_package.egg-info/dependency_links.txt
writing top-level names to example_package.egg-info/top_level.txt
reading manifest file 'example_package.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'example_package.egg-info/SOURCES.txt'
running check
creating example_package-0.1.0
creating example_package-0.1.0/example_package
creating example_package-0.1.0/example_package.egg-info
creating example_package-0.1.0/example_package/cython_files
copying files to example_package-0.1.0...
copying LICENSE -> example_package-0.1.0
copying README.md -> example_package-0.1.0
copying pyproject.toml -> example_package-0.1.0
copying setup.py -> example_package-0.1.0
copying example_package/__init__.py -> example_package-0.1.0/example_package
copying example_package/example.py -> example_package-0.1.0/example_package
copying example_package.egg-info/PKG-INFO -> example_package-0.1.0/example_package.egg-info
copying example_package.egg-info/SOURCES.txt -> example_package-0.1.0/example_package.egg-info
copying example_package.egg-info/dependency_links.txt -> example_package-0.1.0/example_package.egg-info
copying example_package.egg-info/top_level.txt -> example_package-0.1.0/example_package.egg-info
copying example_package/cython_files/file.c -> example_package-0.1.0/example_package/cython_files
copying example_package/cython_files/file2.c -> example_package-0.1.0/example_package/cython_files
Writing example_package-0.1.0/setup.cfg
Creating tar archive
removing 'example_package-0.1.0' (and everything under it)
* Building wheel from sdist
* Creating venv isolated environment...
* Installing packages in isolated environment... (Cython, numpy, setuptools, wheel)
* Getting dependencies for wheel...
Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 351, in <module>
    main()
  File "/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 333, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
    return hook(config_settings)
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
    self.run_setup()
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 10, in <module>
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 970, in cythonize
    module_list, module_metadata = create_extension_list(
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 816, in create_extension_list
    for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):
  File "/private/var/folders/nm/x6s726c95qnc5hzvhgy7pzpm0000gn/T/build-env-j__lzyqr/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 114, in nonempty
    raise ValueError(error_msg)
ValueError: 'example_package/cython_files/file.pyx' doesn't match any files

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_wheel

Project structure:

example_package
|- pyproject.toml
|- setup.py
|- LICENSE, MANIFEST, README.md
|- example_package
   |- __init__.py
   |- example.py
   |- cython_files
      |- file.pyx
      |- file2.pyx
      |- file2.pxd

(Note: There is no __init__.py file in the cython_files directory because it causes a crash when cythonizing and compiling. Instead, I am adding the directory cython_files to sys.path manually.)

setup.py

from setuptools import Extension, setup
from Cython.Build import cythonize
from Cython.Distutils import build_ext

extensions = [Extension('example_package.cython_files.file',
                        ['example_package/cython_files/file.pyx']),
              Extension('example_package.cython_files.file2',
                        ['example_package/cython_files/file2.pyx'])]

extensions = cythonize(extensions)

requirements = ['Cython', 'numpy',]

setup(
    name='example_package',
    version='0.1.0',
    ext_modules=extensions,
    packages=['example_package', 'example_package.cython_files'],
    cmdclass={'build_ext': build_ext},
    package_data = {
        'example_package/cython_files': ['file2.pxd'],
    }
)

pyproject.toml

[build-system]
requires = ['setuptools',
            'wheel',
            'Cython',
            'numpy']
build-backend = "setuptools.build_meta"

example.py

class Example:
    def add_one(self, number):
        return number + 1

    def use_cython(self, number):
        from file import Functions
        f = Functions()
        return f.f(1.)

__ init__ .py

# Manually add 'cython_files' to the path
import sys
from pathlib import Path
sys.path.insert(1, str(Path(__file__).parent / 'cython_files'))

file.pyx (some unimportant calculations)

from file2 cimport File2

cdef class Functions:
    cdef object file2

    def __init__(self):
        self.file2 = File2(5.0)

    cpdef double f(self, double x):
        return x + 1.0

    cpdef double f2(self, double x):
        return self.f(self.file2.add(x))

file2.pyx (some unimportant calculations)

cdef class File2:
    cdef double a

    def __init__(self, a):
        self.a = a

    cpdef double add(self, double b):
        return self.a + b

file2.pxd (some unimportant calculations)

cdef class File2:
    cdef double a

    cpdef double add(self, double b)

Interestingly, I can install the package when I run the following sequence of commands:

  1. Delete the line extensions = cythonize(extensions) from setup.py
  2. Run the command python -m build. This outputs an error.
  3. Put the line extensions = cythonize(extensions) back into the file setup.py where it was.
  4. Run the command python -m build. This succeeds.

However, this is not a permanent solution because I want to just run python -m build.

Lamothe answered 18/9, 2022 at 22:59 Comment(1)
I'm having the exact same error. Have you managed to fix it? – Harvest
H
0

I just managed to fix my occurrence of the very same error. The error happens when the build system calls setup.py to generate the wheel from the sdist. Problem, the *.pyx files are not in the sdist, hence this line in your error:

ValueError: 'example_package/cython_files/file.pyx' doesn't match any files

I fixed the problem by adding the *.pyx file to the sdist. This corresponds to the package_data argument in setup.py.

Secifically in your case, I think changing it to the following would fix the problem:

    package_data = {
        'example_package/cython_files': ['file2.pxd','file.pyx','file2.pyx'],
    }

If this doesn't work then some variation certainly will.

Harvest answered 13/7, 2023 at 9:29 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.