Error while installing python package: llama-cpp-python

Question

I am using Llama to create an application. Previously I used openai but am looking for a free alternative. Based on my limited research, this library provides openai-like api access making it quite easy to add into my prexisting code. However this library has errors while downloading. I tried installing cmake which did not help.

Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      *** scikit-build-core 0.5.1 using CMake 3.27.7 (wheel)
      *** Configuring CMake...
      2023-10-10 21:23:02,749 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
      loading initial cache file C:\Users\ARUSHM~1\AppData\Local\Temp\tmpf1bzj6ul\build\CMakeInit.txt
      -- Building for: NMake Makefiles
      CMake Error at CMakeLists.txt:3 (project):
        Running

         'nmake' '-?'

        failed with:

         The system cannot find the file specified


      CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
      -- Configuring incomplete, errors occurred!

      *** CMake configuration failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

Although not directly related to this question, these are other questions I am unable to get answers for:

1. Does this library use Llama or Llama 2?
2. Will this be secure on a Python Flask Application?

Doodlebug

To answer your first (1.) question: this library wraps llama.cpp (github.com/ggerganov/llama.cpp), which supports a variety of models including both Llama and Llama 2. You can specify the model, including competitors of Llama like Mistral. – Metamorphism

Answer 1

You need to install the desktop c++ block with visual studio to get cmake properly installed.Open the Visual Studio Installer and click Modify, then check Desktop development with C++ and click Modify to start the install. I also recommend the Windows 10 SDK. https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-170 https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/ After that, !pip install llama-cpp-python should build just fine.

Bingham

Just in case you don't have VS installer, you can find it with Download Build Tools from visualstudio.microsoft.com/visual-cpp-build-tools. For me installing from Workloads point Desktop development with C++ was enough. – Hermineherminia

I already have vs code installed. Will this still work? I run Windows btw, and I've had trouble running wsl and vms. – Doodlebug

@Doodlebug vs code and vs are different projects – Adagio

this doesn't work for me – Embryectomy

Answer 2

For this error message

      CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage

The gcc and g++ are NOT installed, and the version of GCC should be over gcc 11 per this issue

---

• Install gcc and g++ under ubuntu

sudo apt update
sudo apt upgrade
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update
sudo apt install gcc-11 g++-11

• Install gcc and g++ under centos

yum install scl-utils 
yum install centos-release-scl
# find devtoolset-11
yum list all --enablerepo='centos-sclo-rh' | grep "devtoolset"

yum install -y devtoolset-11-toolchain

• Install gcc and g++ under Amamzon Linux 2023

sudo dnf install gcc
sudo dnf install g++

Hippel

I am on Windows. Also I have both gcc and g++ installed and on PATH – Doodlebug

@hehe, I did not got one windows environment to test it. Maybe you could reinstall VisualStudio 2022 + C++ building tool and try again? refer github.com/abetlen/llama-cpp-python/issues/54, learn.microsoft.com/en-us/answers/questions/1321159/… – Hippel

Answer 3

I'm trying this commmand in my Ubuntu 22.04 LTS

The error message suggests missing build dependencies for compiling the C++ part of llama-cpp-python. Luckily, Ubuntu provides a convenient package to install these:

sudo apt install build-essential

This command will install gcc (C compiler), g++ (C++ compiler), and other necessary tools for building C/C++ projects.

After this try this command again:

pip install llama-cpp-python

Hope this helps...

Paske

Answer 4

I tried pip install llama-cpp-python --verbose

Then from langchain_community.llms import LlamaCpp

and it worked !

Refer Python bindings for llama.cpp here

Obrien

Answer 5

None of the above mentioned answers worked for me.

Simply by creating a new conda env and installing the package again using pip command solved the error for me.

Senegal