Can not install apache-airflow-providers-mysql: pkg-config error

Question

I'm using Debian/Ubuntu, and I am trying to install an Airflow provider in my Python virtual environment:

$ pip install apache-airflow-providers-mysql
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [24 lines of output]
      /bin/sh: 1: pkg-config: not found
      /bin/sh: 1: pkg-config: not found
      Trying pkg-config --exists mysqlclient
      Command 'pkg-config --exists mysqlclient' returned non-zero exit status 127.
      Trying pkg-config --exists mariadb
      Command 'pkg-config --exists mariadb' returned non-zero exit status 127.
      Traceback (most recent call last):
        File "/home/user/airflow/airflow_env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/user/airflow/airflow_env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/user/airflow/airflow_env/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-a5nk1xf1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-a5nk1xf1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-a5nk1xf1/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 154, in <module>
        File "<string>", line 48, in get_config_posix
        File "<string>", line 27, in find_package_name
      Exception: Can not find valid pkg-config name.
      Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

I'm expecting to see the mysql provider in the airflow connection drop down.

Apogee

Answer 1

specify the MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS environment variables manually, follow the steps below:

1. Open a terminal window.

2. Determine the location of the MySQL header files and libraries. These locations may vary depending on your MySQL installation. You can use the mysql_config command to retrieve this information. Run the following command to find the location of the header files: mysql_config --cflags Run the following command to find the location of the libraries: mysql_config --libs Note down the output of these commands, as you will need them in the next steps.

3. Set the MYSQLCLIENT_CFLAGS environment variable. Replace `` with the path you obtained from the mysql_config --cflags command. Run the following command: export MYSQLCLIENT_CFLAGS="-I"

4. Set the MYSQLCLIENT_LDFLAGS environment variable. Replace `` with the path you obtained from the mysql_config --libs command. Run the following command: export MYSQLCLIENT_LDFLAGS="-L" Note: If your MySQL installation includes multiple libraries, separate each library path with a space.

5. Now, when you install packages that require the MySQL client library, such as mysqlclient for Django, the build process will use the specified CFLAGS and LDFLAGS values to locate the necessary files. You can proceed with your package installation or Django setup, and the specified environment variables will be used to locate the MySQL client libraries.

By manually setting the MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS environment variables, you can ensure that the correct MySQL header files and libraries are used during the installation or compilation process.

I solved the same problem as you using this method

Flann

Helped me, too. We may substitute output of mysql_config to the variables to avoid copy-paste: export MYSQLCLIENT_CFLAGS="$(mysql_config --cflags)" and do same thing for libs export MYSQLCLIENT_LDFLAGS="$(mysql_config --libs)" – Ambulator

it was useful for me , thank you – Godevil

Answer 2

pkg-config: not found is a major hint at the problem and the solution - you're missing build tools and you need to install them.

apt-get install pkg-config build-essential libmysqlclient-dev

note: If you're actually building on a Docker slim image, see: https://mcmap.net/q/747944/-how-to-install-mysqlclient-in-a-python-3-slim-docker-image-without-bloating-the-image

Psalmody

I am using Python 3.10.12. Ubuntu 22. I have installed as per your suggestion and the issue isn't resolved. – Poi

libmysqlclient-dev was missing. Reference -> https://mcmap.net/q/48426/-mysql_config-not-found-when-installing-mysqldb-python-interface – Poi

Answer 3

My pc is macbook，

brew install mysql

can solve it

Deannedeans

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Favoritism

I think the answer is clear enough! – Archaeopteryx

@AlastairMcCormack Its applicable because this question is one of the first google search results, regardless of platform. Their answer saved my morning! – Palish