openai command not found (mac)

Question

I'm trying to follow the fine tuning guide for Openai here.

I ran:

pip install --upgrade openai

Which install without any errors.

But even after restarting my terminal, i still get

zsh: command not found: openai

Here is the output of echo $PATH:

/bin:/usr/bin:/usr/local/bin:/Users/nickrose/Downloads/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

Here is the output of which python:

/usr/bin/python

Any tips for how to fix this? I'm on MacOS Big Sur 11.6.

Goldschmidt

Why exactly are you trying to run openai from the shell, and why should it work? This is a Python package, it is not meant to be run from the shell! – Currey

According to the doc, you should be able to use OpenAI's CLI tool: beta.openai.com/docs/guides/fine-tuning I understand I can also do this using the Python package, I'm just curious why the CLI doesn't work. – Goldschmidt

Here's a quick guide on how to set up the OpenAI CLI on Mac: kevingoedecke.com/2023/04/14/how-to-install-openai-cli-on-macos – Olshausen

Answer 1

Basically pip installs the packages under its related python directory, in a directory called site-packages (most likely, I'm not a python expert tbh). This is not included in the path you provided. First, ask pip to show the location to the package:

pip show openai

The output would be something like this:

Name: openai
Version: 0.22.0
Summary: Python client library for the OpenAI API
Home-page: https://github.com/openai/openai-python
Author: OpenAI
Author-email: [email protected]
License: 
Location: /Users/<USER>/DIR/TO/SOME/PYTHON/site-packages
Requires: numpy, openpyxl, pandas, pandas-stubs, requests, tqdm
Required-by:

So your package will be available in

/Users/<USER>/DIR/TO/SOME/PYTHON/site-packages/openai

Either add /Users/<USER>/DIR/TO/SOME/PYTHON/site-packages/ to your path, or use the complete address to your package, or try to access it using your python:

python -m openai # -m stands for module

To get more information about the -m flag, run python --help.

Update

So as you mentioned in the comments, you get permission denied after you add the directory to your package. This actually means that the package exists, but it's not permitted by your OS to execute. This is the thing you have to do, locate your package, and then:

sudo chmod +x /PATH/TO/script

And the reason you're getting command not found after you use sudo directly with the package, is that you update your path variable in zsh, but when you use sudo, superuser uses sh instead of zsh.

Septicemia

I tried this but got the following: python3 -m openai: /Library/Developer/CommandLineTools/usr/bin/python3: No module named openai.__main__; 'openai' is a package and cannot be directly executed – Goldschmidt

Does your pip use python3? it's more likely you should use python -m openai instead of python3 -m openai. Or, you can do this: python3 -m pip install --upgrade openai, and then: python3 -m openai. – Septicemia

I also tried adding that Location to my PATH variable (i used pip show openai first), and then tried running openai, but I get a zsh: permission denied: openai Then i tried sudo openai, enter my password correctly, and then I get command not found – Goldschmidt

Thanks for the update! I now tried the chmod command, but I still get the "command not found" error after using sudo directly with the package. How do i actually update my PATH variable in sh, rather than in zsh? – Goldschmidt

but I still get the "command not found" error after using sudo directly with the package, you should not use sudo with the package, you should use sudo chmod +x package (If I understood your question correctly). Updating your sh Path is not a good idea in general, but if you couldn't make it with chmod, you can update your profile, or sh resource file if exists. – Septicemia

Tried that as well -- sudo chmod +x /Users/nickrose/Library/Python/3.8/lib/python/site-packages/openai api fine_tunes .create -t fine_tune.jsonl -m davinci (this command is meant to do this part of the tutorial: openai api fine_tunes.create -t <TRAIN_FILE_ID_OR_PATH> -m <BASE_MODEL>). But now I get a bunch of "no such file or directory" errors from chmod for 'api', 'fine_tunes.create', and all the arguments that come after the path/to/openai part – Goldschmidt

@Nick, the chmod command changes the permissions of a script/application. So you'll first need to tell your OS to change the mode of the file to executable (by -x flag) like this: sudo chmod +x Users/nickrose/Library/Python/3.8/lib/python/site-packages/openai (no trailing arguments), and then, execute the app normally (openai api fine_tunes .create -t fine_tune.jsonl -m davinci) – Septicemia

@Goldschmidt @Septicemia So I am stuck in the same issue I tried the same things. I ran the command sudo chmod +x /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/openai and ran the command for openai but still got zsh: command not found: openai – Toxicogenic

@Goldschmidt The "command not found" error is due to ".../site-packages/openai" is a directory... and not an executable. Hence "chmod +x" won't help it, nor does sudo. The openai package. The binary "openai" is located in a different directory for my computer it's /Users/<user>/Library/Python/3.9/bin/openai ; hence you would need to add /Users/<your-user-name>/Library/Python/<Python-Version>/bin to your path or call it using the full path. – Smitherman

What should be the variable name while adding the path? @Septicemia – Sundew

@ShajeelAfzal there’s no need for a new variable, you basically just append a new path to already-existing path. – Septicemia

Answer 2

I was facing similar issue. It might due to global python in your machine is not maching with the pip installation path and it might be installing in some other python folder like in 3.9 and you have 3.10 python version globally set in your Mac.

First install fresh python using homebrew

brew install python

It will install the latest python into your machine. Then try to install openai again using

pip3 install openai

OR using pip (you can try installing using both and see which works as per your system config)

pip install openai

Now

ENJOY a cup of coffee ;)

Wiredraw

Worked for me. However I had to add - brew link --overwrite [email protected] – Aerodonetics

Answer 3

This doesn't answer the question directly but specifies an alternative if you only want to prepare the data set and create the new model for finetunning. It doesn't matter which system you have.

After a lot of struggle I decided it was not worth the hassel to run the cli on my specific machine because of so many different configurations and the mess. My end goal was just to create a model and upload it to OpenAI.

So if someone else stumbles on this post, just use Google Colab. I have also shared one of mine with steps to follow in here.

In case the links don't work in the future I'll list the steps here below as well:

(Step 1)

Set your API key (The already added api key is fake so please replace it with your own):

%env OPENAI_API_KEY=sk-Kz8Weh1234ddgYBmsdfinsdf7ndsfg55532432

(Step 2)

Install the openai package with pip like the following:

!pip install -Uq openai

(Step 3)

Import the openai package like the following:

import openai

(Step 4)

Make sure to upload the promptdata.csv file in the Google Colab folders.

The way to do it is:

1. On the right side you'll see a Hamburger Menu icon click on it.
2. You'll see the "Table of Contents"
3. Click on the last folder icon on the top. If you hover on the icon it says "Files".
4. Now you'll see a folder called "sample_data".
5. Click on the three dots menu for "sample_data" and then select "upload".
6. You should be able to upload your csv file
7. It is not mandatory to upload a csv file. You can also upload any type of TSV, XLSX, JSON or JSONL file as listed by the OpenAI documentation here. But it will always be converted to JSONL file after runnning the below command.

Once you're done uploading the file you can run the below command to prepare your data set which will return you a new JSONL file at the same location where the original file was with all the corrections the tool provides.

!openai tools fine_tunes.prepare_data -f "/content/sample_data/promptdata.csv"

(Step 5)

Run the below command once again after the corrections and it will most likely say "No remediations found".

!openai tools fine_tunes.prepare_data -f "/content/sample_data/promptdata_prepared.jsonl"

(Step 6)

Finally run the below command using the file promptdata_prepared.jsonl and create a model.

!openai api fine_tunes.create -t "/content/sample_data/promptdata_prepared.jsonl"

(Step 7)

Once the model is created note the name of the "Uploaded model"

Toxicogenic

Answer 4

So what happens is that after installing the package there are no actual executables available. That's why you get the error message when you try to execute for example:

openai --help

What i managed to find is that the actual parsing of the commands is done in

/Users/<USER>/DIR_TO_PYTHON/site-packages/openai/_openai_scripts.py

That's just a python script which by default is not executable, so you have to make a workaround of which I find the easiest is creating an executable which basically calls it with the given arguments. Below are the steps which I've done to make it work on "macOS Monterey 12.0.1"

Locate the "openai" package which should be in

/Users/<USER>/DIR_TO_PYTHON/site-packages/

Make sure you are in the "openai" package folder and run

sudo vim /bin/openai

That should create a new file, put in the following command and make sure the path to the file is correct

python3 /Users/<USER>/DIR_TO_PYTHON/site-packages/openai/_openai_scripts.py $@

$@ is for the params that you pass when you call the executable

After saving the file the next step is making it executable which is done with

chmod +x /bin/openai

Last step is adding it to the PATH which is done by adding the file path in /etc/paths and after restarting the terminal, you should have fully working openai command globally

Sherer

Tried sudo vim /bin/openai and entered python3 /Users/<USER>/DIR_TO_PYTHON/site-packages/openai/_openai_scripts.py $@ but now can't exit vim after trying :wq What should I do? – Toxicogenic

Can you send the error? Otherwise It would be a wild guess. – Sherer

I found out that I did not have some access rights on my work laptop. I don't think I solved the issue but used Google Colab to fine tune eventually. – Toxicogenic

Answer 5

I encountered a similar error, but after some searching, I successfully installed it with brew

brew install openai

Walkin

Answer 6

This is what worked for me:

I created an executable in /usr/local/bin that executes the OpenAI Python script:

~/.local/lib/python3.10/site-packages/openai/_openai_scripts.py

Here's the command I used:

 sudo sh -c 'echo python3 /home/<MY_USERNAME>/.local/lib/python3.10/site-packages/openai/_openai_scripts.py \$\@ > /usr/local/bin/openai'

Combative

Answer 7

After trying several other suggestions which didn't work for me on my Mac, I consulted my son who is a python developer. He suggested creating a python virtual environment on my Mac (MacOS: Ventura, 13.4; Chip: M1 Pro) as follows:

python3 -m venv openaiwork (this will create a new openaiwork virtual directory in the current location)

source openaiwork/bin/activate

pip install openai

I also needed to install pandas in this new virtual environment using:

pip install pandas

Once set up as described above, the openai scripts worked like a charm!

Frameup

Answer 8

If you meet:

WARNING: The script openai.exe is installed in 'C:\Users\...\...\local-packages\Python39\Scripts' which is not on PATH.

You should:

export PATH="~\...\...\local-packages\Python39\Scripts:$PATH"

For MacOS and Linux：

export PATH="/path/to/scripts:$PATH" 

For Windows：

setx /m PATH "%PATH%;C:\path\to\scripts" 

Replace C:\path\to\scripts with path to scripts

Voe

Answer 9

Solution for Python3

Run the following command to install the OpenAI CLI

pip3 install --upgrade openai

If you now run openai --help you will still get an error message, telling you command not found.

The next step is to make the Python OpenAI script executable and add it to our path.

Make the OpenAI CLI executable We will have to create a binary file in your local users bin directory (/usr/local/bin). This directory contains all executable binaries that are specific to your user.

sudo sh -c 'echo python3 "$(python3 -m site --user-site)"/openai/_openai_scripts.py \$\@ > /usr/local/bin/openai'

What this command does is the following:

• We get the path to the python site packages directory where all global python packages are installed to: python3 -m site --user-site .
• We extend that path to map to the OpenAI CLI scripts Python script: openai/_openai_scripts.py.
• We then pipe this into a file in the /usr/local/bin directory to use it from anywhere as a shell command.

The last thing we need to do now is to make that file executable:

sudo chmod +x /usr/local/bin/openai

This should be it, you should now be able to run the OpenAI CLI from anywhere in your terminal.

You can test this by doing the following:

openai --help

Which should return all available OpenAI CLI commands.

Thanks to this blog

Cultivation