Pyenv's python is missing bzip2 module

Question

I used pyenv to install python 3.8.2 and to create a virtualenv. In the virtualenv, I used pipenv to install pandas.

However, when importing pandas, I'm getting the following:

  [...]
  File "/home/luislhl/.pyenv/versions/poc-prefect/lib/python3.8/site-packages/pandas/io/common.py", line 3, in <module>
    import bz2
  File "/home/luislhl/.pyenv/versions/3.8.2/lib/python3.8/bz2.py", line 19, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'

After some googling, I found out some people suggesting I rebuild Python from source after installing bzip2 library in my system.

However, after trying installing it with sudo dnf install bzip2-devel I see that I already had it installed.

As far as I know, pyenv builds python from source when installing some version. So, why wasn't it capable of including the bzip2 module when building?

How can I manage to rebuild Python using pyenv in order to make bzip2 available? I'm in Fedora 30

Thanks in advance

UPDATE I tried installing another version of python with pyenv in verbose mode, to see the compilation output.

There is this message in the end of the compilation:

WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?

But as I stated before, I checked I already have bzip2 installed in my system. So I don't know what to do.

Silvanasilvano

Add some information about your operating system and also try pip install bzip2 in the virtual env and add the output of that. – Breslau

bzip2 doesn't seem to be a valid package: pypi.org/project/bzip2 – Silvanasilvano

Sorry, I guess it is bz2 – Breslau

bz2 doesn't work either: Could not find a version that matches bz2 – Silvanasilvano

Answer 1

On Ubuntu 22 LTS

Missing Library Problem in Python Installation with Pyenv

Before the fix:

$> pyenv install 3.11.0

command result:

pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz


Installing Python-3.11.0...
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
WARNING: The Python lzma extension was not compiled. Missing the lzma lib?

TLDR;

Recipe to fix:

sudo apt-get install build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev

Result

After the fix:

$> pyenv install 3.11.0

Command result:

pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
Installing Python-3.11.0...
Installed Python-3.11.0 to /home/user/.pyenv/versions/3.11.0

Lorinalorinda

This command line alone would not be enough (at least for me). You should also install tkinter with this: sudo apt-get install python-tk python3-tk tk-dev And add the ncurses library to your previous cmd: sudo apt-get install build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev libncurses-dev – Philemon

I had the same problem with pyenv doctor and the fix solved the issue – Guanabara

See suggested-build-environment in pyenv documentation. – Unbelieving

I had the same issue trying installing python 3.8 using pyenv also I have to included tkinter packages as @AlexNeaKameni mentioned. Ubuntu 22 lts – Amidst

if there is a warning: WARNING: The Python tkinter extension was not compiled and GUI subsystem has been detected. Missing the Tk toolkit?, try apt install tk-dev. – Multitudinous

This command also misses libncurses-dev . – Prosecute

Answer 2

On macOS Big Sur, to get pyenv ( via homebrew ) to work I had to install zlib and bzip2 via homebrew and then add the exports in my ~/.zshrc ( or ~/.bashrc for bash I guess). The answer above by luislhl leads the way to my solution.

brew install zlib bzip2

#Add the following to your ~/.zshrc
# For pyenv to build
export LDFLAGS="-L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include -I/usr/local/opt/bzip2/include"

# Then the install worked
pyenv install 3.7.9

Blackhead

There is a typo in the CPPFLAGS. The include folder should be bzip2, not bzip3 – Aaronaaronic

Answer 3

For Python 3.12 on Ubuntu 22 LTS.
This should install all the libs pyenv needs

sudo apt-get install build-essential python-tk python3-tk tk-dev  zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev libncurses-dev

pyenv install 3.12

Eth

Answer 4

Ok, I have found the solution after some time. It was simple, but I took some time to realize it.

It turns out the problem was the bzip2-devel I had installed was a 32-bit version.

The compilation process was looking for the 64-bit one, and didn't find it. So I had to specifically install the 64-bit version:

sudo dnf install bzip2-devel-1.0.6-29.fc30.x86_64

Silvanasilvano

Answer 5

Thanks, that helped, just with small modification in ~/.zshrc:

export LDFLAGS="-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/bzip2/lib"
export CPPFLAGS="-I/opt/homebrew/opt/zlib/include -I/opt/homebrew/opt/bzip2/include"

and then pyenv install 3.7.9

---

Apple M1, macOS 11.1 20C69 arm64;

➜ brew --version           
Homebrew 2.7.1
Homebrew/homebrew-core (git revision ad6fd8; last commit 2021-01-05)
Homebrew/homebrew-cask (git revision 5c3de; last commit 2021-01-04)

---

but this didn't help with No module named '_ctypes' on M1 :(

Poulos

I think that for the sake of completeness it is probably better always let HomeBrew find out its own path and use these declarations export LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" and export CPPFLAGS="-I$(brew --prefix zlib)/include -I$(brew --prefix bzip2)/include" – Moment

Answer 6

For me, on Fedora 39, to get pyenv to work I had to install the following packages

sudo dnf install bzip2-devel ncurses-devel libffi-devel readline-devel tk-devel libsqlite3x-devel

Libove